Some binaries and man pages were not included in rpm builds or debian
builds. This adds those artifacts to ceph.spec.in
debian/ceph-common.install and debian/radosgw.install.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
librdkafka submodule
Vendors librdkafka v2.12.1 as a submodule (default) and teaches the
notification teuthology task to set up Kafka 4.x KRaft brokers. The
distro-shipped librdkafka 1.6.1 cannot SCRAM-auth against Kafka 4.x
brokers (#75900); the bundled version fixes that. Build with
WITH_SYSTEM_RDKAFKA=ON to opt back into the distro library (floor 2.6.1).
follwinf changes needed with teh submodule:
* build kafka_stub only in case of system librdkafka
* use the same header location in submodule and system
* supress compilation warning to avoid -WError issues
2.12.1 matches Fedora(44)'s current librdkafka-devel, so the vendored copy
tracks a version that distro packagers have already vetted rather than
whichever release happens to be latest upstream.
The teuthology version matrix picks one of {3.9.2, 4.1, 4.2, 4.3} per run.
3.9.2 covers the last Zookeeper-based release for the migration cohort.
4.1/4.2/4.3 track Apache's current three supported minor lines and are
resolved to the latest patch at test time via dlcdn.apache.org/kafka/, so
new patches within a supported minor need no PR update. A new minor line
(4.4) requires renaming one YAML.
* new java version (17) is needed for kafka 4.2 and up (we wil use it on
all version)
* kerberos task need to run before the kafka install task
After pulling this branch, fetch the new submodule:
git submodule update --init src/librdkafka
Fixes: https://tracker.ceph.com/issues/77336
Signed-off-by: ShreeJejurikar <shreemj8@gmail.com>
Co-authored-by: Yuval Lifshitz <ylifshit@ibm.com>
When building with DWZ enabled, the debian packaging fails with:
```
dh_dwz: error: Aborting due to earlier error
```
Running the dwz command manually reveals the root cause:
```
$ dwz -mdebian/ceph-osd-crimson/usr/lib/debug/.dwz/x86_64-linux-gnu/ceph-osd-crimson.debug \
-M/usr/lib/debug/.dwz/x86_64-linux-gnu/ceph-osd-crimson.debug -- \
debian/ceph-osd-crimson/usr/bin/ceph-osd-crimson \
debian/ceph-osd-crimson/usr/bin/crimson-store-nbd
dwz: debian/ceph-osd-crimson/usr/bin/ceph-osd-crimson: Too many DIEs, not optimizing
dwz: Too few files for multifile optimization
```
The dwz tool has a limit on the number of DWARF DIEs (Debug Information
Entries) it can process. The ceph-osd-crimson binary, being a large C++
executable with extensive template usage, exceeds this limit, causing
dwz to exit with status 1 and fail the build.
This change excludes only ceph-osd-crimson from dwz processing using
the -X flag, allowing other binaries in the package to still benefit
from DWARF compression while avoiding the build failure.
Please note, we always disable DWZ in ceph-build's ceph-dev-pipeline.
Fixes: https://tracker.ceph.com/issues/78303
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
CherryPy and packaging are listed as runtime dependencies of
ceph-mgr-modules-core, but none of the modules shipped in that package
import either one. CherryPy is used by the dashboard, cephadm, and
prometheus modules, which list it in their own .requires; packaging is
not imported by any shipped mgr code. Both entries therefore pull
python3-cherrypy and python3-packaging into ceph-mgr-modules-core for no
reason.
Drop them. The remaining entries (natsort, requests, python-dateutil,
pyyaml) match what the core modules import; prettytable is declared
separately in debian/control.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
BlueStore's PMEMDevice backend targets byte-addressable persistent
memory exposed as a DAX device, in practice Intel Optane DC persistent
memory. Intel Optane has been discontinued since 2021 and there is no
mainstream replacement, so the backend has effectively no hardware left
to run on. It was always experimental and has seen no functional change
since the DML/DSA offload landed in 2022; every commit to it since has
been a formatting sweep.
Keeping it builds in a chain of dependencies that nothing else needs:
libpmem, libndctl and libdaxctl, plus dml for the DSA offload that only
works on a Sapphire Rapids CPU. Remove the backend and everything tied
to it.
RBD persistent write-back cache (WITH_RBD_RWL) still uses PMDK's
libpmemobj, so the pmdk submodule, WITH_SYSTEM_PMDK, Buildpmdk and
Findpmdk stay. WITH_SYSTEM_PMDK now depends on WITH_RBD_RWL alone, and
Buildpmdk no longer builds ndctl support.
- remove the WITH_BLUESTORE_PMEM option, HAVE_BLUESTORE_PMEM and
HAVE_LIBDML
- delete src/blk/pmem/PMEMDevice.{cc,h} and the pmem block_device_t
- drop the pmem value from the bdev_type option
- remove the --bluestore-pmem vstart flag
- drop the libpmem, libndctl and libdaxctl build deps from debian and
rpm, keeping libpmemobj for RWL
- delete the Finddml, Findndctl and Finddaxctl cmake modules
- remove the DSA/DML and ndctl documentation
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
The mgr module install rule already excluded tests/*, but that glob only
dropped the files inside tests/ and left an empty tests/ directory in the
installed tree. The ci/ directories, which hold Dockerfiles, e2e test
scripts, and cluster specs used only for upstream CI pipelines, were not
excluded at all and were shipped by ceph-mgr-dashboard and ceph-mgr-rook.
Change the install pattern from tests/* to tests so the whole directory
is pruned, and add a ci pattern. Both patterns live in the exclude list
shared by every mgr module install rule (the module list, dashboard, and
rook), so this covers all of them in one place.
This supersedes the per-package excludes added in 8a2e0eb2fc and
d74b88500b, so drop them from ceph.spec.in and debian/rules.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
The ceph-mgr-dashboard and ceph-mgr-rook packages install their entire
mgr module directory, which includes a ci/ subdirectory containing
Dockerfiles, e2e test scripts, and cluster specs used only for upstream
CI pipelines. ceph-mgr-cephadm similarly ships a tests/ directory with
Python unit tests. These files have no runtime purpose on a deployed
system and should not be shipped in the binary packages.
Exclude mgr/cephadm/tests, mgr/dashboard/ci, and mgr/rook/ci via
dh_install --exclude.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
in order to silence the warnings like:
```
W: rbd-nbd: old-fsf-address-in-copyright-file
```
let's refere to the email address of FSF, instead of its old postal
address.
See https://lintian.debian.org/tags/old-fsf-address-in-copyright-file.html
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
ceph-mgr-modules-core was split into per-module packages so that users
only need to install what they actually use. To ease migration for
existing deployments that want the full former set, add a meta-package
ceph-mgr-modules-standard that pulls in all modules which were
previously bundled in ceph-mgr-modules-core.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
cli_api is a new module in this release (not previously shipped in
ceph-mgr-modules-core), so it gets its own package without any
Breaks/Replaces or Obsoletes against the old monolithic package.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
diskprediction-local depends on python3-prettytable and osd-support
depends on python3-cherrypy3; both need to be declared explicitly now
that these modules are separate packages.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
ceph-mgr-modules-core has historically bundled always-on modules
together with optional ones, forcing users to install modules and their
dependencies even when they have no use for them. Split each optional
module into its own package so users and distributions can install only
what they need.
ceph-mgr-modules-core is trimmed to the 10 always-on modules defined
in src/mon/MgrMonitor.cc: balancer, crash, devicehealth, orchestrator,
pg_autoscaler, progress, rbd_support, status, telemetry, volumes.
Each optional module now follows the pattern of ceph-mgr-k8sevents and
ceph-mgr-rook.
New packages carry Breaks/Replaces: ceph-mgr-modules-core (<< 21.0.0)
for proper file ownership transfer on upgrade.
The split also exposes cross-module Python dependencies: modules
co-installed in ceph-mgr-modules-core could freely import each other,
but once separated into individual packages those imports require
explicit Depends entries. Now the inter-dependencies are properly
reflected in debian/control.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
override_dh_strip enumerates each binary package explicitly. It was not
updated when ceph-osd was split into the ceph-osd-classic and
ceph-osd-crimson implementation packages, so the OSD binaries in those
two packages are shipped unstripped (ceph-osd-crimson installs at ~4.6
GiB) and their -dbg packages are left empty.
Add the missing dh_strip invocations so the OSD binaries are stripped
and their debug symbols land in the corresponding -dbg packages, as is
already done for every other binary package.
Fixes: a37b5b5bde
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
The ceph-osd-crimson package already lists ${shlibs:Depends} in its
Depends field, which generates the correct libprotobuf dependency for
the target distribution at build time (e.g. libprotobuf32t64 on
Trixie/Noble). The hardcoded libprotobuf23 entry is redundant and
breaks installations on distributions where protobuf ships under a
different package name.
See also ab4c5daead which added this
runtime dep.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
5843c6b04b gated the build on /etc/redhat-release because gRPC devel
libs weren't packaged on Debian yet. They are now (libgrpc++-dev,
protobuf-compiler-grpc), so we can finally bring Debian derivatives on
par with Fedora/RHEL and ship the NVMe-oF gateway monitor client there
too.
This change:
- drops the /etc/redhat-release sniff and unconditionally enables
WITH_NVMEOF_GATEWAY_MONITOR_CLIENT.
- adds libgrpc++-dev and protobuf-compiler-grpc to debian/control's
Build-Depends, plus a ceph-mon-client-nvmeof / -dbg pair so the
binary actually gets packaged.
- adds a pkg-config fallback for gRPC discovery. Jammy's libgrpc++-dev
(1.30.2) ships no cmake config files [1], so find_package(gRPC CONFIG
REQUIRED) fails at configure time. Noble's libgrpc++-dev (1.51.1)
does ship them [2], as do RHEL/Rocky packages. We now try cmake config
first (QUIET) and fall back to pkg_check_modules(IMPORTED_TARGET
grpc++) when it isn't found.
- patches PkgConfig::GRPCPP to carry protobuf::libprotobuf as an
interface dependency. grpc++.pc omits protobuf from its Requires, so
gateway.pb.cc (which calls GOOGLE_PROTOBUF_VERIFY_VERSION) would fail
to link on the pkg-config path. The cmake-config gRPC::grpc++ declares
this dependency properly; we match that behaviour with
target_link_libraries(PkgConfig::GRPCPP INTERFACE protobuf::libprotobuf).
- applies HAVE_ABSEIL only on the cmake-config path (Noble, RHEL/Rocky),
where gRPC links system absl. Without it, opentelemetry-cpp's private
absl (inline namespace otel_v1) collides with system absl (inline
namespace debian7) in any TU that includes both tracer.h and
grpcpp/grpcpp.h, giving "reference to base_internal is ambiguous".
On the pkg-config path (Jammy's gRPC 1.30.2) libabsl-dev is not
installed, so HAVE_ABSEIL must be skipped there.
[1] https://packages.ubuntu.com/jammy/amd64/libgrpc-dev/filelist
[2] https://packages.ubuntu.com/noble/amd64/libgrpc-dev/filelist
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
the build already produces usr/share/man/man8/cephfs-top.8, and the
fedora spec ships it via the cephfs-top package. on debian,
debian/cephfs-top.install was missing it, so the file was sitting in
debian/tmp unclaimed and dh_missing complained:
dh_missing: warning: usr/share/man/man8/cephfs-top.8 exists in
debian/tmp but is not installed to anywhere
claim the manpage in debian/cephfs-top.install so it lands in the
cephfs-top deb. silences the warning and gives debian users the same
`man cephfs-top` fedora users already have.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
These deps should be included as part of the dependencies of the
python-onelogin-saml2 package. The names of the packages have changed
between jammy (22.04) and resolute (26.04) and it's not convenient (or
possible?) in debian packaging to have different dependencies for
different versions of the OS. Since they'll be pulled in as needed by
onelogin package, we can just remove these.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
osd/rados/rgw/cephfs: Modernize cls interface with compile time safety
Reviewed-by: Bill Scales <bill_scales@uk.ibm.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Adam Emerson <aemerson@ibm.com>
The osd code was not present, the radosacl tool did nothing useful and the scratchtoolpp
was testing something that doesn't exist.
key_value_store is optionally compiled into the key store, but seems to serve no useful purpose.
Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>
Status Quo:
Librados currently uses string-based identifiers for RADOS class method calls
(exec) in both ObjectReadOperation and ObjectWriteOperation. The API does
not distinguish between methods that modify object state and those that
are read-only.
Problem:
This lack of semantic enforcement has become a critical safety gap with the
introduction of EC Direct Reads. Because the EC direct read path is optimized
for performance, it may process the same read-only operation multiple times
(e.g., across different shards or during retries).
Previously, while technically a violation of API semantics, a "read-that-writes"
would often function correctly because operations were processed in-order and
without retries on the read path. However, in the context of EC direct reads,
executing a non-idempotent write through the read path poses a silent and
critical risk to data integrity.
Solution:
Introduce a template-based trait system to encode method semantics into the
API. By tagging methods with traits (MethodRead or MethodWrite), the compiler
now prevents state-modifying methods from being added to ObjectReadOperation.
This ensures that only side-effect-free methods reach the EC direct read
path, eliminating the risk of accidental double-writes at the source.
Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>
Add libjerasure-dev to Build-Depends and enable WITH_SYSTEM_JERASURE
so that the Debian package builds ec_jerasure and ec_shec plugins
against the distribution-provided libraries instead of the vendored
copies.
libjerasure-dev pulls in libgf-complete-dev, so both libraries are
covered by a single build dependency.
Add libjerasure2 to ceph-base's Depends for the runtime shared
libraries. libjerasure2 in turn depends on libgf-complete1t64.
The explicit dependency is needed because dh_shlibdeps excludes
erasure-code plugins from automatic dependency scanning (they are
loaded via dlopen at runtime).
The distro-packaged jerasure and gf-complete are from the same
upstream git snapshots (2017/04/10) as ceph's vendored copies.
gf-complete source is byte-identical. jerasure differs only in
additional null-pointer checks and error handling in jerasure.c,
plus an extra galois_uninit_field() function that ceph does not use.
All function signatures ceph depends on are present and compatible.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
distutils was deprecated in Python 3.10 (PEP 632) and removed in
Python 3.12. The `python3-distutils` package no longer exists in
Debian Trixie (Python 3.13) or Ubuntu 24.04+ (Python 3.12).
The only runtime reference was in `debian/ceph-mgr.requires`, already
cleaned up by 3fb3f892aa. This override is now dead code, hence no
installed file declares a runtime dependency on `distutils`, so
`dh_python3` never resolves it. Removing it prevents a latent
uninstallable-dependency bug if `distutils` were accidentally
reintroduced in a `.requires` file.
Fixes: https://tracker.ceph.com/issues/75901
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
RGW's Kafka notification path accepts and propagates the verify-ssl
topic attribute, but rgw_kafka.cc never applied it to librdkafka.
This patch configures enable.ssl.certificate.verification from
verify-ssl and separates cached connections by verify-ssl setting.
Fixes: https://tracker.ceph.com/issues/55790
Signed-off-by: 邓伟键 <emmmvkdeng@gmail.com>
mgr: isolated CherryPy to prevent global state sharing
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
as the modules are now being loaded onto the main interpreter (see
https://github.com/ceph/ceph/pull/66244), the
cherrypy is getting hit with an issue where its global state is being
affecting all the modules updating the cherrypy config simultaneously in
the same tree.
So i am adding a CherryPyMgr which manages all the independent servers
that will be created across all modules. This CherryPyMgr will create
its own server instances by utilizing cherrypy's WSGI Server and
eliminates the global state sharing. Each module or app can create their
own tree and start an adapter which will open an independent server for
that app.
- also added a method to update the config in place so CORS urls can be
configured without restarting servers.
Fixes: https://tracker.ceph.com/issues/74643, https://tracker.ceph.com/issues/74543, https://tracker.ceph.com/issues/74980
Signed-off-by: Nizamudeen A <nia@redhat.com>
The `BaseController` auto-imports the packaged `mgr/dashboard/controllers/smb.py`
file, which in turn wants to import `smb.enums` etc. which is part of the `smb`
package which is missing from `debian/ceph-mgr-modules-core.install`, thus
missing in the package. The missing module causes an exception
`ModuleNotFoundError: No module named 'smb'` on mgr instances when running a
ceph tentacle cluster installed from debian packages.
See: https://tracker.ceph.com/issues/74268
Signed-off-by: Roland Sommer <rol@ndsommer.de>
Replace 'setup.py install' with 'pip install --use-pep517' to fix
Cython compilation failures and eliminate deprecation warnings.
Problem Statement:
The build process for Cython modules involves preprocessing .pyx files
(e.g., generating rbd_processed.pyx from rbd.pyx) and then cythonizing
with specific compiler_directives. The previous approach using separate
'setup.py build' and 'setup.py install' commands caused this failure:
```
Error compiling Cython file:
------------------------------------------------------------
...
"""
name = cstr(name, 'name')
cdef:
rados_ioctx_t _ioctx = convert_ioctx(ioctx)
char *_name = name
librbd_progress_fn_t _prog_cb = &no_op_progress_callback
^
------------------------------------------------------------
rbd_processed.pyx:781:44: Cannot assign type 'int (*)(uint64_t, uint64_t, void *) except? -1' to 'librbd_progress_fn_t'. Exception values are incompatible. Suggest adding 'noexcept' to type 'int (uint64_t, uint64_t, void *) except? -1'.
```
This occurs because:
1. 'setup.py build build_ext' successfully preprocesses and cythonizes
with compiler_directives from setup.py's cythonize() call
2. 'setup.py install' internally triggers a rebuild that:
- Regenerates the preprocessed .pyx files
- Re-runs cythonize() through Cython.Distutils.build_ext
- Does NOT apply the compiler_directives from setup.py
- Fails on the regenerated files missing required directives
New Options Explained:
`--use-pep517`:
Addresses deprecation warning:
```
DEPRECATION: Building 'rados' using the legacy setup.py bdist_wheel
mechanism, which will be removed in a future version. pip 25.3 will
enforce this behaviour change.
```
Uses the modern PEP 517 build backend which:
- Performs a single build pass with all compiler_directives applied
- Prevents the implicit rebuild that caused CompileError
- Future-proofs against pip 25.3+ which will require this
`--no-build-isolation`:
Ensures that environment variables set by CMake are respected:
- CC, LDSHARED (compiler toolchain)
- CPPFLAGS, LDFLAGS (compilation flags)
- CYTHON_BUILD_DIR, CEPH_LIBDIR (build paths)
Without this flag, pip would create an isolated build environment
that ignores these critical build settings.
`--no-deps`:
Prevents pip from attempting to install Python dependencies listed
in setup.py's install_requires. All dependencies are managed by
CMake and the distribution's package manager, not pip.
`--ignore-installed`:
Addresses installation error when DESTDIR is set:
```
ERROR: Could not install packages due to an OSError: [Errno 13]
Permission denied: '/usr/lib/python3/dist-packages/rados-2.0.0.egg-info'
OSError: [Errno 18] Invalid cross-device link:
'/usr/lib/python3/dist-packages/rados-2.0.0.egg-info' -> '/tmp/pip-uninstall-...'
```
This error occurs because pip detects an existing system installation
and tries to uninstall it before installing to DESTDIR. With
--ignore-installed, pip skips the uninstall step and directly installs
to the DESTDIR staging directory, which is the correct behavior for
packaging.
Removed Options:
`--install-layout=deb`:
This Debian-specific patch to 'setup.py install' is no longer needed.
Modern pip automatically detects the distribution and uses the correct
layout (dist-packages on Debian, site-packages on RPM distros).
`--single-version-externally-managed`:
This option was specific to 'setup.py install' to prevent egg
installation. With pip, this is handled automatically.
`--record /dev/null`:
No longer needed as pip manages installation records internally.
`egg_info --egg-base`:
Not needed with pip as metadata is generated automatically during
the build process.
Not added option:
`--root-user-action=ignore`: not added
In this change, we installing a python module using pip with
`fakeroot` before packaging it. But pip warned:
```
Error: WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behavior with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
```
But we use fakeroot on purpose, this option could have been added to
silence this warning. But it is not available in all supported pip
versions. see
2e1112a814
New environmental variable:
`DEB_PYTHON_INSTALL_LAYOUT=deb` is conditionally applied when packaging
for debian-derivative distributions. As pip does not support
`--install-layout` option. Since debian patches pip so it installs Python
modules into /usr/local/lib instead of /usr/lib where debian dh_install
helper looks for the content to be packaged, so we have to enforce the
debian layout using the environmental variable.
Working Directory Change:
Changed from `CMAKE_CURRENT_SOURCE_DIR` to `CMAKE_CURRENT_BINARY_DIR` to
keep pip's temporary files and logs in the build directory rather than
polluting the source tree.
Additional Dependencies:
Since the build process uses pip and creates a wheel distribution,
we need to add `pip` and `wheel` Python modules as build dependencies.
Python moduels packaging:
- with `--use-pep517`, pip creates .dist-info directoires as per PEP-517
instead of .egg-info, so we need to package the new metadata directory.
Future Improvements
We considered implementing a custom `build_templates` command or using
setuptools' `sub_commands` mechanism to avoid regenerating `*_processed.pyx`
files on every build (tracking dependencies via file modification times or
hash-based checks). However, to keep `setup.py` simple and maintainable,
we've deferred this optimization for future work. The current solution
using `pip install --use-pep517` ensures correct builds without additional
complexity.
This solution works correctly for both Debian and RPM packaging workflows,
both of which use DESTDIR-based staged installations.
Fixes: 719b749846
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
Test scripts like qa/tasks/cephfs/mount.py expect the ip command to be
available in the container environment. Without it, tests fail with:
```
/bin/bash: line 1: ip: command not found
File "/ceph/qa/tasks/cephfs/mount.py", line 96, in cleanup_stale_netnses_and_bridge
p = remote.run(args=['ip', 'netns', 'list'],
...
teuthology.exceptions.CommandFailedError: Command failed with status 127: 'ip netns list'
```
Add iproute2 to the debian package build dependencies when the
<pkg.ceph.check> build profile is enabled. This ensures the package is
available during container-based builds, since buildcontainer-setup.sh
→ script/run-make.sh → install-deps.sh → debian/control → generated
dependency package chain respects build profiles configured via
`FOR_MAKE_CHECK` and `WITH_CRIMSON` environment variables set in
Dockerfile.build.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
