Some binaries and man pages were not included in rpm builds or debian
builds. This adds those artifacts to ceph.spec.in
debian/ceph-common.install and debian/radosgw.install.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
librdkafka submodule
Vendors librdkafka v2.12.1 as a submodule (default) and teaches the
notification teuthology task to set up Kafka 4.x KRaft brokers. The
distro-shipped librdkafka 1.6.1 cannot SCRAM-auth against Kafka 4.x
brokers (#75900); the bundled version fixes that. Build with
WITH_SYSTEM_RDKAFKA=ON to opt back into the distro library (floor 2.6.1).
follwinf changes needed with teh submodule:
* build kafka_stub only in case of system librdkafka
* use the same header location in submodule and system
* supress compilation warning to avoid -WError issues
2.12.1 matches Fedora(44)'s current librdkafka-devel, so the vendored copy
tracks a version that distro packagers have already vetted rather than
whichever release happens to be latest upstream.
The teuthology version matrix picks one of {3.9.2, 4.1, 4.2, 4.3} per run.
3.9.2 covers the last Zookeeper-based release for the migration cohort.
4.1/4.2/4.3 track Apache's current three supported minor lines and are
resolved to the latest patch at test time via dlcdn.apache.org/kafka/, so
new patches within a supported minor need no PR update. A new minor line
(4.4) requires renaming one YAML.
* new java version (17) is needed for kafka 4.2 and up (we wil use it on
all version)
* kerberos task need to run before the kafka install task
After pulling this branch, fetch the new submodule:
git submodule update --init src/librdkafka
Fixes: https://tracker.ceph.com/issues/77336
Signed-off-by: ShreeJejurikar <shreemj8@gmail.com>
Co-authored-by: Yuval Lifshitz <ylifshit@ibm.com>
Add a build target that builds standalone RGW in addition to normal RGW,
and add an RPM for it that only installs the necessary bits.
Signed-off-by: Daniel Gryniewicz <dang@fprintf.net>
BlueStore's PMEMDevice backend targets byte-addressable persistent
memory exposed as a DAX device, in practice Intel Optane DC persistent
memory. Intel Optane has been discontinued since 2021 and there is no
mainstream replacement, so the backend has effectively no hardware left
to run on. It was always experimental and has seen no functional change
since the DML/DSA offload landed in 2022; every commit to it since has
been a formatting sweep.
Keeping it builds in a chain of dependencies that nothing else needs:
libpmem, libndctl and libdaxctl, plus dml for the DSA offload that only
works on a Sapphire Rapids CPU. Remove the backend and everything tied
to it.
RBD persistent write-back cache (WITH_RBD_RWL) still uses PMDK's
libpmemobj, so the pmdk submodule, WITH_SYSTEM_PMDK, Buildpmdk and
Findpmdk stay. WITH_SYSTEM_PMDK now depends on WITH_RBD_RWL alone, and
Buildpmdk no longer builds ndctl support.
- remove the WITH_BLUESTORE_PMEM option, HAVE_BLUESTORE_PMEM and
HAVE_LIBDML
- delete src/blk/pmem/PMEMDevice.{cc,h} and the pmem block_device_t
- drop the pmem value from the bdev_type option
- remove the --bluestore-pmem vstart flag
- drop the libpmem, libndctl and libdaxctl build deps from debian and
rpm, keeping libpmemobj for RWL
- delete the Finddml, Findndctl and Finddaxctl cmake modules
- remove the DSA/DML and ndctl documentation
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
python3_pkgversion picked the native python3 package name on RHEL7, where
python3 was python34/python36 before 7.6 (cea9d18c). RHEL7 and RHEL8 are
gone, so on every supported distro (el9, el10, fedora, suse, openEuler) it
is always 3.
Replace python%{python3_pkgversion} with python3 and drop the fallback
default. No supported distro overrides it to build against a non-default
python flavor, so that capability goes too.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
Many Python deps were split across %if suse / %if rhel branches only
because openSUSE spells them CamelCase (PrettyTable, PyYAML, Sphinx,
CherryPy, Routes, Jinja2) and fedora/rhel/openEuler lowercase. The
python3dist() provides resolve the same on el9, el10, openEuler and
openSUSE, so those splits collapse to single lines.
Spell the deps python%{python3_pkgversion}dist(), matching the spec's
existing python%{python3_pkgversion}-Foo convention. Version constraints
and feature guards are preserved.
The dist name is the PyPI project name, not always the rpm name minus
python3-: python3-dateutil provides python3dist(python-dateutil), and
python3-saml provides python3dist(python3-saml).
Left as package names: the bare interpreter; python3-devel and numpy-devel,
which have no python3dist() provide; the build toolchain (setuptools,
Cython, pip, wheel); and the ceph-internal python3-* subpackages, which are
versioned = release deps.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
Library BuildRequires used per-distro -devel package names, some split
across %if suse / %if rhel branches only because the names differ. The
pkgconfig() virtual provides resolve identically on every rpm distro; the
spec already relies on them for fuse3, systemd and udev.
Switch every library dependency that ships a .pc to pkgconfig(), preserving
version constraints and feature guards. This collapses the suse-vs-rhel
naming splits (nss, keyutils, openssl, ldap, ibverbs, rdmacm, lz4, thrift,
libcryptopp) and the dedicated split blocks (nlohmann_json, lttng-ust,
babeltrace, expat) into single lines, and folds the duplicated xmlsec1 devel
packages together.
Left as package names: libaio and xfsprogs ship no reliable .pc; gperftools
carries version and tcmalloc-variant logic; numa is unconditional on rhel
but crimson-only on suse; and the xmlsec1 runtime backend plugins are
runtime test deps, not build-time pkgconfig() deps.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
RHEL 8 and CentOS 8 left the supported test matrix in 2023 (449db416,
7a1dce1e). RHEL 8 has also been unbuildable since CMakeLists.txt began
requiring Python >= 3.9: it ships python 3.6 by default, so cmake refuses
to configure. The spec still carried the el8 conditionals, so assume
rhel >= 9 and drop them:
- remove branches reachable only on rhel <= 8: the gperftools 2.6.1
BuildRequires, the tracker-36508 tcmalloc-libs requirement, and the
dataclasses backport, which also targeted that dead python 3.6
- collapse "rhel >= 8" and "rhel >= 9" to plain "rhel"; with the floor at
9 this is a no-op for rhel 9/10, fedora, suse and openEuler
- keep the rhel 9-vs-10 checks (== 9, < 10, >= 10)
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
ceph-osd-crimson and ceph-osd-classic call update-alternatives in their
%posttrans and %preun scriptlets but don't depend on it. declare it as a
scriptlet dependency so the binary is there when they run.
Fixes: https://tracker.ceph.com/issues/77323
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
a37b5b5bde added %preun scriptlets that use ${_sbindir}, which is
shell syntax rather than an rpm macro, so it expands to empty at run
time and the scriptlet runs "/update-alternatives", failing on
uninstall/upgrade with:
/var/tmp/rpm-tmp.K1fvm3: line 2: /update-alternatives: No such file or directory
error: %preun(ceph-osd-crimson-2:20.3.0-5054.g33c1d671.el9.x86_64) scriptlet failed, exit status 127
Error in PREUN scriptlet in rpm package ceph-osd-crimson.
use %{_sbindir}, like the %posttrans --install lines already do, so it
expands to /usr/sbin/update-alternatives at build time.
Fixes: https://tracker.ceph.com/issues/77323
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
87e233bb26 added an unconditional
"Requires: c-ares >= 1.28.0" to ceph-osd-crimson: seastar links
ares_query_dnsrec, which c-ares only grew in 1.28, and the libcares.so.2
SONAME doesn't carry the version so rpm can't infer the floor itself.
But the floor only earns its place where the build links the symbol
against a newer c-ares than the runtime has, and that's an EL thing.
el10's minors cross 1.28 under one $releasever (10.1 ships 1.25, 10.2
ships 1.34), so a builder rolls to 1.34 while a frozen 10.1 node stays on
1.25; without the floor the rpm installs there and the osd then crashes
on the missing symbol. el9 builds the legacy ares_query path and doesn't
need it at all.
Fedora and SUSE don't have the skew: one c-ares per release, built and
run against the same one, so the auto libcares.so.2 dep covers them. So
pin it only on el10+, arch-qualified with %{?_isa}.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
Seastar's DNS stack uses ares_query_dnsrec when built against c-ares
>= 1.28 (ARES_VERSION >= 0x011c00). Only ceph-osd-crimson links that
path; classic-osd does not, so add the version floor on the crimson
subpackage only.
Rocky Linux 10 shaman builds use docker.io/rockylinux/rockylinux:10
(os-release 10.1), but dnf builddeps resolve against the live Rocky 10
BaseOS/AppStream repos, which track the newest minor and install
c-ares-devel/c-ares 1.34.6. CMake links ceph-osd-crimson against that
library. Teuthology nodes are provisioned as Rocky 10.1 and install only
the requested Ceph packages without a full distro upgrade, so their
baseline c-ares stays at 1.25.0 (< 1.28, no ares_query_dnsrec). Install
succeeds but OSD startup fails with "undefined symbol: ares_query_dnsrec".
Require c-ares >= 1.28 on ceph-osd-crimson so dnf upgrades to a suitable
libcares (1.34.6 is already in Rocky 10.1 baseos) or fails cleanly at
install. Ubuntu crimson CI does not show this mismatch: the same LTS is
used for building and testing, and maintainers do not bump upstream
package versions across an LTS lifecycle (only cherry-picked fixes), so
build-time and runtime libc-ares stay aligned.
Signed-off-by: Kautilya Tripathi <kautilya.tripathi@ibm.com>
cephadm's zipapp imports yaml and jinja2. When the package is built
with cephadm_bundling and RPM-sourced deps (--without cephadm_pip_deps),
only BuildRequires were listed for jinja2 and no runtime Requires were
declared for PyYAML or Jinja2, so minimal "dnf install cephadm" could
fail with ModuleNotFoundError. The unbundled cephadm case also lacked
PyYAML.
Add matching BuildRequires/Requires in the rpm-bundle branch and
Requires (plus SUSE naming for Jinja2/PyYAML) in the no-bundle branch.
Fixes: https://tracker.ceph.com/issues/75389
Signed-off-by: Kobi Ginon <kginon@redhat.com>
ceph-mgr-modules-core was split into per-module packages so that users
only need to install what they actually use. To ease migration for
existing deployments that want the full former set, add a meta-package
ceph-mgr-modules-standard that pulls in all modules which were
previously bundled in ceph-mgr-modules-core.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
cli_api is a new module in this release (not previously shipped in
ceph-mgr-modules-core), so it gets its own package without any
Breaks/Replaces or Obsoletes against the old monolithic package.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
ceph-mgr-modules-core has historically bundled always-on modules
together with optional ones, forcing users to install modules and their
dependencies even when they have no use for them. Split each optional
module into its own package so users and distributions can install only
what they need.
ceph-mgr-modules-core is trimmed to the 10 always-on modules defined
in src/mon/MgrMonitor.cc: balancer, crash, devicehealth, orchestrator,
pg_autoscaler, progress, rbd_support, status, telemetry, volumes.
Each optional module now follows the pattern of ceph-mgr-k8sevents and
ceph-mgr-rook.
New packages carry Obsoletes: ceph-mgr-modules-core < 21.0.0 for
proper upgrade path.
The split also exposes cross-module Python dependencies: modules
co-installed in ceph-mgr-modules-core could freely import each other,
but once separated into individual packages those imports require
explicit Requires entries. Now the inter-dependencies are expressed
properly in ceph.spec.in.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
Define %ceph_mgr_module_scripts() to emit the identical %post/%postun
pair for each optional mgr module package, replacing the 5 existing
hand-written copies (dashboard, diskprediction-local, rook, k8sevents,
cephadm) with a single call site per package. Subsequent commits that
introduce new mgr module packages can use the macro from the start.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
Add a new --with pypkg option that passes WITH_PYPKG to cmake.
This allows building with the new (experimental) python packaging
support. If this proves useful a future change can consider enabling
by default in some conditions.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
osd/rados/rgw/cephfs: Modernize cls interface with compile time safety
Reviewed-by: Bill Scales <bill_scales@uk.ibm.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Adam Emerson <aemerson@ibm.com>
i don't see golang-github-prometheus available for centos anymore and
other distro's as well. And different package provides the promtool in
different distro's so instead of identifying all the corresponding
packages and its name, replacing the package name with binary path so it
works across distros without distro specific conditions
Some build failures are captured in our internal runs recently
https://github.com/rhcs-dashboard/ceph-dev/actions/runs/24298848427/job/70949666821
Signed-off-by: Nizamudeen A <nia@redhat.com>
The osd code was not present, the radosacl tool did nothing useful and the scratchtoolpp
was testing something that doesn't exist.
key_value_store is optionally compiled into the key store, but seems to serve no useful purpose.
Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>
Status Quo:
Librados currently uses string-based identifiers for RADOS class method calls
(exec) in both ObjectReadOperation and ObjectWriteOperation. The API does
not distinguish between methods that modify object state and those that
are read-only.
Problem:
This lack of semantic enforcement has become a critical safety gap with the
introduction of EC Direct Reads. Because the EC direct read path is optimized
for performance, it may process the same read-only operation multiple times
(e.g., across different shards or during retries).
Previously, while technically a violation of API semantics, a "read-that-writes"
would often function correctly because operations were processed in-order and
without retries on the read path. However, in the context of EC direct reads,
executing a non-idempotent write through the read path poses a silent and
critical risk to data integrity.
Solution:
Introduce a template-based trait system to encode method semantics into the
API. By tagging methods with traits (MethodRead or MethodWrite), the compiler
now prevents state-modifying methods from being added to ObjectReadOperation.
This ensures that only side-effect-free methods reach the EC direct read
path, eliminating the risk of accidental double-writes at the source.
Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>
RGW's Kafka notification path accepts and propagates the verify-ssl
topic attribute, but rgw_kafka.cc never applied it to librdkafka.
This patch configures enable.ssl.certificate.verification from
verify-ssl and separates cached connections by verify-ssl setting.
Fixes: https://tracker.ceph.com/issues/55790
Signed-off-by: 邓伟键 <emmmvkdeng@gmail.com>
