Commit Graph

580 Commits

Author SHA1 Message Date
Kefu Chai
757d192b43
Merge pull request #69726 from tchaikov/wip-drop-bluestore-pmem
blk,build: drop BlueStore PMEM support

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
2026-07-28 13:27:17 +08:00
ShreeJejurikar
efe54523a4 rgw: support kafka 4.x brokers
librdkafka submodule

Vendors librdkafka v2.12.1 as a submodule (default) and teaches the
notification teuthology task to set up Kafka 4.x KRaft brokers. The
distro-shipped librdkafka 1.6.1 cannot SCRAM-auth against Kafka 4.x
brokers (#75900); the bundled version fixes that. Build with
WITH_SYSTEM_RDKAFKA=ON to opt back into the distro library (floor 2.6.1).
follwinf changes needed with teh submodule:
* build kafka_stub only in case of system librdkafka
* use the same header location in submodule and system
* supress compilation warning to avoid -WError issues

2.12.1 matches Fedora(44)'s current librdkafka-devel, so the vendored copy
tracks a version that distro packagers have already vetted rather than
whichever release happens to be latest upstream.

The teuthology version matrix picks one of {3.9.2, 4.1, 4.2, 4.3} per run.
3.9.2 covers the last Zookeeper-based release for the migration cohort.
4.1/4.2/4.3 track Apache's current three supported minor lines and are
resolved to the latest patch at test time via dlcdn.apache.org/kafka/, so
new patches within a supported minor need no PR update. A new minor line
(4.4) requires renaming one YAML.
* new java version (17) is needed for kafka 4.2 and up (we wil use it on
  all version)
* kerberos task need to run before the kafka install task

After pulling this branch, fetch the new submodule:

    git submodule update --init src/librdkafka

Fixes: https://tracker.ceph.com/issues/77336

Signed-off-by: ShreeJejurikar <shreemj8@gmail.com>
Co-authored-by: Yuval Lifshitz <ylifshit@ibm.com>
2026-07-22 10:29:04 +00:00
Kefu Chai
80b9067286 blk,build: drop BlueStore PMEM support
BlueStore's PMEMDevice backend targets byte-addressable persistent
memory exposed as a DAX device, in practice Intel Optane DC persistent
memory. Intel Optane has been discontinued since 2021 and there is no
mainstream replacement, so the backend has effectively no hardware left
to run on. It was always experimental and has seen no functional change
since the DML/DSA offload landed in 2022; every commit to it since has
been a formatting sweep.

Keeping it builds in a chain of dependencies that nothing else needs:
libpmem, libndctl and libdaxctl, plus dml for the DSA offload that only
works on a Sapphire Rapids CPU. Remove the backend and everything tied
to it.

RBD persistent write-back cache (WITH_RBD_RWL) still uses PMDK's
libpmemobj, so the pmdk submodule, WITH_SYSTEM_PMDK, Buildpmdk and
Findpmdk stay. WITH_SYSTEM_PMDK now depends on WITH_RBD_RWL alone, and
Buildpmdk no longer builds ndctl support.

- remove the WITH_BLUESTORE_PMEM option, HAVE_BLUESTORE_PMEM and
  HAVE_LIBDML
- delete src/blk/pmem/PMEMDevice.{cc,h} and the pmem block_device_t
- drop the pmem value from the bdev_type option
- remove the --bluestore-pmem vstart flag
- drop the libpmem, libndctl and libdaxctl build deps from debian and
  rpm, keeping libpmemobj for RWL
- delete the Finddml, Findndctl and Finddaxctl cmake modules
- remove the DSA/DML and ndctl documentation

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-05 08:37:21 +08:00
Kefu Chai
0ac9881841
Merge pull request #68989 from tchaikov/wip-slim-mgr-module
debian,rpm: split ceph-mgr-modules-core into per-module packages

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2026-06-05 12:53:45 +08:00
Kefu Chai
3b90401c0c
Merge pull request #68817 from tchaikov/nvme-of-mon-client
cmake,debian: enable ceph-mon-client-nvmeof on Debian derivatives

Reviewed-by: Dan Mick <dan.mick@redhat.com>
2026-06-03 15:00:43 +08:00
Kefu Chai
f2e29ed8bf debian, rpm: add ceph-mgr-modules-standard meta-package
ceph-mgr-modules-core was split into per-module packages so that users
only need to install what they actually use. To ease migration for
existing deployments that want the full former set, add a meta-package
ceph-mgr-modules-standard that pulls in all modules which were
previously bundled in ceph-mgr-modules-core.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-01 16:00:26 +08:00
Kefu Chai
fab568213a debian, rpm: add ceph-mgr-cli-api package
cli_api is a new module in this release (not previously shipped in
ceph-mgr-modules-core), so it gets its own package without any
Breaks/Replaces or Obsoletes against the old monolithic package.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-01 16:00:26 +08:00
Kefu Chai
7ed409e9c7 debian: fix missing python3 deps for diskprediction-local and osd-support
diskprediction-local depends on python3-prettytable and osd-support
depends on python3-cherrypy3; both need to be declared explicitly now
that these modules are separate packages.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-01 16:00:26 +08:00
Kefu Chai
5d3161367b debian: split ceph-mgr-modules-core into per-module packages
ceph-mgr-modules-core has historically bundled always-on modules
together with optional ones, forcing users to install modules and their
dependencies even when they have no use for them. Split each optional
module into its own package so users and distributions can install only
what they need.

ceph-mgr-modules-core is trimmed to the 10 always-on modules defined
in src/mon/MgrMonitor.cc: balancer, crash, devicehealth, orchestrator,
pg_autoscaler, progress, rbd_support, status, telemetry, volumes.
Each optional module now follows the pattern of ceph-mgr-k8sevents and
ceph-mgr-rook.

New packages carry Breaks/Replaces: ceph-mgr-modules-core (<< 21.0.0)
for proper file ownership transfer on upgrade.

The split also exposes cross-module Python dependencies: modules
co-installed in ceph-mgr-modules-core could freely import each other,
but once separated into individual packages those imports require
explicit Depends entries. Now the inter-dependencies are properly
reflected in debian/control.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-01 16:00:26 +08:00
Kefu Chai
bb3f03d0f8
Merge pull request #68258 from tchaikov/wip-with-system-jerasure
cmake: support building with system jerasure and gf-complete

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2026-05-26 17:53:02 +08:00
Dan Mick
e232d16236
Merge pull request #68602 from phlogistonjohn/jjm-bwc-u26
script/build-with-container: add distro references for ubuntu 26.04
2026-05-13 17:24:25 -07:00
Kefu Chai
88d4c1ce12 debian: drop explicit libprotobuf dependency from ceph-osd-crimson
The ceph-osd-crimson package already lists ${shlibs:Depends} in its
Depends field, which generates the correct libprotobuf dependency for
the target distribution at build time (e.g. libprotobuf32t64 on
Trixie/Noble). The hardcoded libprotobuf23 entry is redundant and
breaks installations on distributions where protobuf ships under a
different package name.

See also ab4c5daead which added this
runtime dep.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-05-12 17:24:28 +08:00
Kefu Chai
62cf01d0de cmake,debian: enable ceph-mon-client-nvmeof on Debian derivatives
5843c6b04b gated the build on /etc/redhat-release because gRPC devel
libs weren't packaged on Debian yet. They are now (libgrpc++-dev,
protobuf-compiler-grpc), so we can finally bring Debian derivatives on
par with Fedora/RHEL and ship the NVMe-oF gateway monitor client there
too.

This change:

- drops the /etc/redhat-release sniff and unconditionally enables
  WITH_NVMEOF_GATEWAY_MONITOR_CLIENT.
- adds libgrpc++-dev and protobuf-compiler-grpc to debian/control's
  Build-Depends, plus a ceph-mon-client-nvmeof / -dbg pair so the
  binary actually gets packaged.
- adds a pkg-config fallback for gRPC discovery. Jammy's libgrpc++-dev
  (1.30.2) ships no cmake config files [1], so find_package(gRPC CONFIG
  REQUIRED) fails at configure time. Noble's libgrpc++-dev (1.51.1)
  does ship them [2], as do RHEL/Rocky packages. We now try cmake config
  first (QUIET) and fall back to pkg_check_modules(IMPORTED_TARGET
  grpc++) when it isn't found.
- patches PkgConfig::GRPCPP to carry protobuf::libprotobuf as an
  interface dependency. grpc++.pc omits protobuf from its Requires, so
  gateway.pb.cc (which calls GOOGLE_PROTOBUF_VERIFY_VERSION) would fail
  to link on the pkg-config path. The cmake-config gRPC::grpc++ declares
  this dependency properly; we match that behaviour with
  target_link_libraries(PkgConfig::GRPCPP INTERFACE protobuf::libprotobuf).
- applies HAVE_ABSEIL only on the cmake-config path (Noble, RHEL/Rocky),
  where gRPC links system absl. Without it, opentelemetry-cpp's private
  absl (inline namespace otel_v1) collides with system absl (inline
  namespace debian7) in any TU that includes both tracer.h and
  grpcpp/grpcpp.h, giving "reference to base_internal is ambiguous".
  On the pkg-config path (Jammy's gRPC 1.30.2) libabsl-dev is not
  installed, so HAVE_ABSEIL must be skipped there.

[1] https://packages.ubuntu.com/jammy/amd64/libgrpc-dev/filelist
[2] https://packages.ubuntu.com/noble/amd64/libgrpc-dev/filelist

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-05-08 21:28:41 +08:00
John Mulligan
e16cd9342c debian: remove xmlsec1 deps from control file
These deps should be included as part of the dependencies of the
python-onelogin-saml2 package. The names of the packages have changed
between jammy (22.04) and resolute (26.04) and it's not convenient (or
possible?) in debian packaging to have different dependencies for
different versions of the OS.  Since they'll be pulled in as needed by
onelogin package, we can just remove these.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2026-05-05 10:46:40 -04:00
Kefu Chai
11e45a45f5 debian: build with system jerasure and gf-complete
Add libjerasure-dev to Build-Depends and enable WITH_SYSTEM_JERASURE
so that the Debian package builds ec_jerasure and ec_shec plugins
against the distribution-provided libraries instead of the vendored
copies.

libjerasure-dev pulls in libgf-complete-dev, so both libraries are
covered by a single build dependency.

Add libjerasure2 to ceph-base's Depends for the runtime shared
libraries. libjerasure2 in turn depends on libgf-complete1t64.
The explicit dependency is needed because dh_shlibdeps excludes
erasure-code plugins from automatic dependency scanning (they are
loaded via dlopen at runtime).

The distro-packaged jerasure and gf-complete are from the same
upstream git snapshots (2017/04/10) as ceph's vendored copies.
gf-complete source is byte-identical. jerasure differs only in
additional null-pointer checks and error handling in jerasure.c,
plus an extra galois_uninit_field() function that ceph does not use.
All function signatures ceph depends on are present and compatible.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-04-09 08:27:41 +08:00
邓伟键
595a753771 rgw/kafka: honor verify-ssl and separate cached connections
RGW's Kafka notification path accepts and propagates the verify-ssl
topic attribute, but rgw_kafka.cc never applied it to librdkafka.

This patch configures enable.ssl.certificate.verification from
verify-ssl and separates cached connections by verify-ssl setting.

Fixes: https://tracker.ceph.com/issues/55790
Signed-off-by: 邓伟键 <emmmvkdeng@gmail.com>
2026-03-27 10:31:04 +08:00
Kefu Chai
91de0e1933 cmake: migrate Python module installation from setup.py to pip
Replace 'setup.py install' with 'pip install --use-pep517' to fix
Cython compilation failures and eliminate deprecation warnings.

Problem Statement:
The build process for Cython modules involves preprocessing .pyx files
(e.g., generating rbd_processed.pyx from rbd.pyx) and then cythonizing
with specific compiler_directives. The previous approach using separate
'setup.py build' and 'setup.py install' commands caused this failure:

```
Error compiling Cython file:
------------------------------------------------------------
...
        """
        name = cstr(name, 'name')
        cdef:
            rados_ioctx_t _ioctx = convert_ioctx(ioctx)
            char *_name = name
            librbd_progress_fn_t _prog_cb = &no_op_progress_callback
                                            ^
------------------------------------------------------------

rbd_processed.pyx:781:44: Cannot assign type 'int (*)(uint64_t, uint64_t, void *) except? -1' to 'librbd_progress_fn_t'. Exception values are incompatible. Suggest adding 'noexcept' to type 'int (uint64_t, uint64_t, void *) except? -1'.
```

This occurs because:
1. 'setup.py build build_ext' successfully preprocesses and cythonizes
   with compiler_directives from setup.py's cythonize() call
2. 'setup.py install' internally triggers a rebuild that:
   - Regenerates the preprocessed .pyx files
   - Re-runs cythonize() through Cython.Distutils.build_ext
   - Does NOT apply the compiler_directives from setup.py
   - Fails on the regenerated files missing required directives

New Options Explained:

`--use-pep517`:
  Addresses deprecation warning:
  ```
    DEPRECATION: Building 'rados' using the legacy setup.py bdist_wheel
    mechanism, which will be removed in a future version. pip 25.3 will
    enforce this behaviour change.
  ```
  Uses the modern PEP 517 build backend which:
  - Performs a single build pass with all compiler_directives applied
  - Prevents the implicit rebuild that caused CompileError
  - Future-proofs against pip 25.3+ which will require this

`--no-build-isolation`:
  Ensures that environment variables set by CMake are respected:
  - CC, LDSHARED (compiler toolchain)
  - CPPFLAGS, LDFLAGS (compilation flags)
  - CYTHON_BUILD_DIR, CEPH_LIBDIR (build paths)

  Without this flag, pip would create an isolated build environment
  that ignores these critical build settings.

`--no-deps`:
  Prevents pip from attempting to install Python dependencies listed
  in setup.py's install_requires. All dependencies are managed by
  CMake and the distribution's package manager, not pip.

`--ignore-installed`:
  Addresses installation error when DESTDIR is set:
  ```
    ERROR: Could not install packages due to an OSError: [Errno 13]
    Permission denied: '/usr/lib/python3/dist-packages/rados-2.0.0.egg-info'

    OSError: [Errno 18] Invalid cross-device link:
    '/usr/lib/python3/dist-packages/rados-2.0.0.egg-info' -> '/tmp/pip-uninstall-...'
  ```
  This error occurs because pip detects an existing system installation
  and tries to uninstall it before installing to DESTDIR. With
  --ignore-installed, pip skips the uninstall step and directly installs
  to the DESTDIR staging directory, which is the correct behavior for
  packaging.

Removed Options:

`--install-layout=deb`:
  This Debian-specific patch to 'setup.py install' is no longer needed.
  Modern pip automatically detects the distribution and uses the correct
  layout (dist-packages on Debian, site-packages on RPM distros).

`--single-version-externally-managed`:
  This option was specific to 'setup.py install' to prevent egg
  installation. With pip, this is handled automatically.

`--record /dev/null`:
  No longer needed as pip manages installation records internally.

`egg_info --egg-base`:
  Not needed with pip as metadata is generated automatically during
  the build process.

Not added option:
`--root-user-action=ignore`: not added
  In this change, we installing a python module using pip with
  `fakeroot` before packaging it. But pip warned:
  ```
  Error: WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behavior with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
  ```
  But we use fakeroot on purpose, this option could have been added  to
  silence this warning. But it is not available in all supported pip
  versions. see
  2e1112a814

New environmental variable:
`DEB_PYTHON_INSTALL_LAYOUT=deb` is conditionally applied when packaging
for debian-derivative distributions. As pip does not support
`--install-layout` option. Since debian patches pip so it installs Python
modules into /usr/local/lib instead of /usr/lib where debian dh_install
helper looks for the content to be packaged, so we have to enforce the
debian layout using the environmental variable.

Working Directory Change:

Changed from `CMAKE_CURRENT_SOURCE_DIR` to `CMAKE_CURRENT_BINARY_DIR` to
keep pip's temporary files and logs in the build directory rather than
polluting the source tree.

Additional Dependencies:

Since the build process uses pip and creates a wheel distribution,
we need to add `pip` and `wheel` Python modules as build dependencies.

Python moduels packaging:

- with `--use-pep517`, pip creates .dist-info directoires as per PEP-517
instead of .egg-info, so we need to package the new metadata directory.

Future Improvements

We considered implementing a custom `build_templates` command or using
setuptools' `sub_commands` mechanism to avoid regenerating `*_processed.pyx`
files on every build (tracking dependencies via file modification times or
hash-based checks). However, to keep `setup.py` simple and maintainable,
we've deferred this optimization for future work. The current solution
using `pip install --use-pep517` ensures correct builds without additional
complexity.

This solution works correctly for both Debian and RPM packaging workflows,
both of which use DESTDIR-based staged installations.

Fixes: 719b749846
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-01-29 08:45:50 +08:00
Kefu Chai
599922aa58 debian/control: add iproute2 to build dependencies
Test scripts like qa/tasks/cephfs/mount.py expect the ip command to be
available in the container environment. Without it, tests fail with:

```
  /bin/bash: line 1: ip: command not found

  File "/ceph/qa/tasks/cephfs/mount.py", line 96, in cleanup_stale_netnses_and_bridge
    p = remote.run(args=['ip', 'netns', 'list'],
  ...
  teuthology.exceptions.CommandFailedError: Command failed with status 127: 'ip netns list'
```

Add iproute2 to the debian package build dependencies when the
<pkg.ceph.check> build profile is enabled. This ensures the package is
available during container-based builds, since buildcontainer-setup.sh
→ script/run-make.sh → install-deps.sh → debian/control → generated
dependency package chain respects build profiles configured via
`FOR_MAKE_CHECK` and `WITH_CRIMSON` environment variables set in
Dockerfile.build.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2025-12-24 14:17:24 +08:00
Kefu Chai
3eac3d9f71
Merge pull request #66568 from Matan-B/wip-matanb-crimson-obselte
debian,ceph.spec: fix ceph-osd upgrade conflicts

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2025-12-12 09:01:20 +08:00
Matan Breizman
9f1c8f9447 debian,ceph.spec: fix ceph-osd upgrade conflicts
With https://github.com/ceph/ceph/pull/65782 merged, upgrading ceph-osd
would need to replace the previous ceph-osd existing on the machine.
Otherwise, we won't be able to symlink the newly installed package:
```
2025-12-05T21:09:20.472 INFO:teuthology.orchestra.run.smithi077.stdout:
Installing       : ceph-osd-classic-2:20.3.0-4434.g8611241d.el9.x86_6
24/87
2025-12-05T21:09:20.478 INFO:teuthology.orchestra.run.smithi077.stdout:
Running scriptlet: ceph-osd-classic-2:20.3.0-4434.g8611241d.el9.x86_6
24/87
2025-12-05T21:09:20.479
INFO:teuthology.orchestra.run.smithi077.stdout:failed to link
/usr/bin/ceph-osd -> /etc/alternatives/ceph-osd: /usr/bin/ceph-osd
exists and it is not a symlink
```

Note: debian/control ceph-osd-classic already had Replace and Breaks:
      - Breaks is replaced with Conflicts to not allow coexistence.
      - Release version is bumped up to be relevant for latest main

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2025-12-11 11:56:46 +00:00
Thomas Lamprecht
a6ab6c6385 debian/control: record python3-packaging dependency for ceph-volume
Commit 0985e201 added `from packaging import version` to
ceph_volume/util/encryption.py. On Debian and its derivatives, the
packaging module is provided by the python3-packaging package.

Add python3-packaging to ceph-volume's runtime dependencies to ensure
this import is satisfied.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Max R. Carrara <m.carrara@proxmox.com>
2025-12-05 12:17:33 +08:00
Matan Breizman
b891223a0f debian,ceph.spec: add ceph-osd-crimson as dependency
a37b5b5bde added ceph-osd-crimson as dependency using OR requirement.
Once ceph-osd-classic dependency is satisfied, ceph-osd-crimson package must be added manually.
In order to not add crimson packages manually (See "Containerfile"
removed lines) - mark osd-crimson as dependency if with_crimson is
enabled for rpm and if pkg.ceph.crimson build profile is enabled for debian.

With this change, users won't be required to obtain ceph-osd-crimson
package manually when relevant. Similarly to ceph-osd-classic being
a dependency for ceph-osd.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2025-11-19 13:14:09 +00:00
Kefu Chai
2aa9727b63
Merge pull request #65936 from tchaikov/wip-build-cephadm-with-deb
cephadm, debian/rules: Use system packages for cephadm bundled dependencies

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2025-11-14 21:17:55 +08:00
Kefu Chai
a37b5b5bde debian,ceph.spec: split ceph-osd into shared base and implementation packages
Previously, ceph-osd packaging had two mutually exclusive flavors that
could only be built one at a time: one with classic OSD and another
with crimson OSD. Both provided /usr/bin/ceph-osd, making them
impossible to coexist and confusing from a user perspective.
This commit restructures the packaging to enable both implementations
to coexist on the same system:

- ceph-osd: Contains shared components (systemd units, sysctl configs,
  common executables like ceph-erasure-code-tool) and depends on exactly
  one OSD implementation
- ceph-osd-classic: Contains the classic OSD implementation binary and
  classic-specific tools
- ceph-osd-crimson: Contains the crimson OSD implementation binary and
  crimson-specific tools

The two implementation packages install different sets of file, so they
don't conflict with each other anymore, and both depend on ceph-osd for
shared resources.

Changes:

Debian packaging:

- Revert e5f00d2f
- Add ceph-osd-crimson package
- Add Recommends: ceph-osd-classic to prefer classic on upgrades
- Add Replaces/Breaks for smooth upgrades from old monolithic package
- Create separate .install files for crimson and classic osd packages
  Enforce exact version matching using ${binary:Version}

RPM packaging:

- Use rich dependencies for OR requirement (classic or crimson)
- Add Recommends: ceph-osd-classic for upgrade preference

Upgrade behavior:

Users upgrading from older versions will automatically get
ceph-osd-classic due to the Recommends directive, maintaining
backward compatibility. Users can explicitly choose crimson by
installing ceph-osd-crimson, which will coexist with classic.
Switching between implementations is supported via standard package
operations, with the alternatives system ensuring /usr/bin/ceph-osd
always points to the active implementation.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2025-11-12 18:07:29 +00:00
Kefu Chai
25680021ee debian: Use system packages for cephadm bundled dependencies
Configure the Debian build to use CEPHADM_BUNDLED_DEPENDENCIES=deb,
which instructs the cephadm build script to bundle dependencies from
system-installed Debian packages instead of downloading from PyPI.

This change addresses build failures in restricted network environments
where Debian build tools do not permit internet access. By leveraging
the Debian package support added in commit 9378a2988e1, the build now
uses python3-markupsafe, python3-jinja2, and python3-yaml packages
that are already installed as build dependencies.

This approach mirrors the existing RPM packaging workflow, ensuring
consistent behavior across different distribution package formats.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2025-11-12 11:59:14 +08:00
Kefu Chai
929b47f838 debian/control: Add libxsimd-dev build dependency for vendored Arrow
In commit e8460cbd, we introduced the "pkg.ceph.arrow" build profile to
support building with system Arrow packages. However, neither Debian nor
Ubuntu currently ships Arrow packages.

Since WITH_RADOSGW_SELECT_PARQUET is always enabled in debian/rules,
Arrow support is required for all builds. When the pkg.ceph.arrow profile
is not selected, the build uses vendored Arrow. With the recent change to
use AUTO mode for xsimd detection, Arrow will attempt to find system xsimd
>= 9.0.1. Adding libxsimd-dev as a build dependency ensures it's available
for Arrow to detect and use, reducing build time on supported distributions.

On distributions with insufficient xsimd versions (< 9.0.1), Arrow will
automatically fall back to its bundled version.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2025-10-15 15:49:09 +08:00
Dan Mick
fa6c38c1d0 debian/control: install libnuma-dev unconditionally
The arm64-only module uadk needs numa.h to build; nothing else
ensures it's available.  Make it an unconditional ceph build
dependency on behalf of the arm64 build.

Fixes: https://tracker.ceph.com/issues/72594
Signed-off-by: Dan Mick <dan.mick@redhat.com>
2025-08-18 15:22:02 -05:00
Nizamudeen A
8acf8f7e4f mgr/dashboard: disable saml2 tests when the dep is not there
Signed-off-by: Nizamudeen A <nia@redhat.com>
2025-07-01 13:29:43 +05:30
Thomas Lamprecht
03a93dbb18 debian: radosgw: add media-types packages as alternative for mime-support
Back in 2020 the '/etc/mime.types' file was split out into it's own
leaner 'media-types' Debian package [0], the previous 'mime-support'
was kept as transitional package to handle upgrades, but it did its
job and isn't included in Debian Trixie repos anymore.

So accept both, the new 'media-types' package or the legacy
'mime-support' one as valid dependency for radosgw, this ensures one
can install radosgw on newer Debian based releases while keeping it
also working for older ones.

[0]: 4d6ff19de5

Fixes: https://tracker.ceph.com/issues/71545
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2025-06-03 22:43:59 +02:00
Pedro Gonzalez Gomez
567c9e7667 mgr/dashboard: add xlmtodict import and fix lifecycle get request
Fixes: https://tracker.ceph.com/issues/70128
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2025-03-13 21:50:18 +01:00
Afreen Misbah
f65b00ea8a mgr/dashboard: Improve sso role mapping
Fixes https://tracker.ceph.com/issues/70366

- adds support for ISV
- using jmespath expression for fetching roles from payload
- added roles_path parameter in sso enable command as optional argument
- modified roles mapper for sso login

Signed-off-by: Afreen Misbah <afreen@ibm.com>
2025-03-10 16:42:03 +05:30
Yuval Lifshitz
e443d75b4f test/rgw/lua: install luarocks
since it is only a suggestion for debian.
this is a regression from: https://github.com/ceph/ceph/pull/60666

Fixes: https://tracker.ceph.com/issues/69473

Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
2025-01-16 14:57:09 +00:00
Kefu Chai
29f31e67c6
Merge pull request #60666 from ThomasLamprecht/debian-fix-librgw2-lua-dependencies
debian/control: fix overly broad lua dependency declaration for librgw2 package

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2025-01-07 22:19:43 +08:00
Venky Shankar
955bf28fe4 Merge PR #58376 into main
* refs/pull/58376/head:
	Temporarily change the libcephfs dependencies
	proxy: Add the design document
	proxy: Add the proxy to the deb builds
	proxy: Add the proxy to the rpm builds
	Initial version of the libcephfs proxy

Reviewed-by: Sachin Prabhu <sp@spui.uk>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2024-12-13 13:24:05 +05:30
Thomas Lamprecht
cc74b76f3e d/control: fix lua dependencies for librgw2
One can attach lua scripts as sort of hooks to implement dynamic
checks or transformations of RGW requests since Ceph Pacific. Thus, a
lua library is now required for base support and optionally one can
use the luarocks deployment and management system for Lua modules to
use more advanced scripts/modules.

With commit 46500cace6 ("rgw/test/lua: add lua integration tests
suite") the dependency relations got cleaned up, as the respective
entries were missing completely from debian/control.

But that commit is pulling in much more than required due to adding
the devel package `liblua5.3-dev` instead of the library-only
`liblua5.3-0` one, and having `luarocks` as hard dependency compared
to an optional Suggests. Fixing that avoids pulling in a whole
build/compiler/autotools/... stack with 65 new packages just when one
wants to use librgw2 or python3-rgw for simple RGW requests, or just
needs the ceph-common package, which pulls in librgw2 transitively.

This was reported by prolific community member Neobin on the Proxmox
forum [0], and then discussed on the original PR, adding the
dependencies [1].

[0]: https://forum.proxmox.com/threads/156433/post-715148
[1]: https://github.com/ceph/ceph/pull/52931#issuecomment-2441253989

Fixes: https://tracker.ceph.com/issues/68873
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-11-07 17:42:42 +01:00
Ernesto Puerta
96ec7badb8
mgr/restful: remove deprecated module
Detailed changes:
* Remove `restful` mgr module dir,
* Remove Python depedencies (`pecan`, `werkzeug`) from ceph.spec and
  debian control,
* Remove docs,
* Remove associated QA tests,
* Update vstart.

Fixes: https://tracker.ceph.com/issues/47066
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2024-10-28 14:17:18 +01:00
Xavi Hernandez
c7c982cdb6 proxy: Add the proxy to the deb builds
Signed-off-by: Xavi Hernandez <xhernandez@gmail.com>
2024-09-26 12:22:47 +02:00
Effi Ofer
dc8eed68ce librbd/migration: add nbd stream
Co-authored-by: Ilya Dryomov <idryomov@gmail.com>
Signed-off-by: Effi Ofer <effio@il.ibm.com>
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2024-09-06 14:14:12 +02:00
Kefu Chai
729fd8e25f ceph-volume: add "packaging" to install_requires
in 0985e201, "packaging" was introduced as a runtime dependency of
ceph-volume, and `ceph.spec.in` was updated accordingly to note
this new dependency. but the debian packaging was not updated.

in 80edcd40, the missing dependency was added to debian/control as
one of ceph-volume's runtime dependency.

but dh_python3 is able to figure out the dependencies by reading
the egg's metadata of the ceph-volume python module. and as a
python project, ceph-volume is using its `setup.py` for
tracking its dependencies.

so in order to be more consistent, and keep all of its dependencies
in one place, let's move this dependency to setup.py . as the
packagings in both distros are able to figure the dependencies
from egg-info.

see also
- https://manpages.debian.org/testing/dh-python/dh_python3.1.en.html#dependencies
- https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_201x/#_automatically_generated_dependencies

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-08-14 07:04:00 +08:00
Thomas Lamprecht
80edcd40e8 debian pkg: record python3-packaging dependency for ceph-volume
Since commit 0985e20134 ("ceph-volume: use 'no workqueue' options
with dmcrypt") the python "packaging" module is used to parse the
cryptsetup version output, but the debian packaging was not updated to
record that new dependency.

So simply record this in the d/control file, adding a <pkg>.requires
file seemed to not really winning us anything here.

Fixes: https://tracker.ceph.com/issues/67290
Fixes: 0985e20134
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-07-31 10:44:58 +02:00
Yuri Weinstein
bf30425f5e
Merge pull request #55409 from cybozu/add-ceph-exporter-to-deb-package
debian: add ceph-exporter package

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
2024-03-20 08:40:57 -07:00
Samuel Just
ab4c5daead debian/control: add runtime protobuf dependency for crimson-osd inherited from seastar
Signed-off-by: Samuel Just <sjust@redhat.com>
2024-03-14 13:59:45 -07:00
Kefu Chai
080e386098 debian/control: replace <tab> with spaces
to be consistent with the rest of this file

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-02-22 23:35:45 +08:00
Yuri Weinstein
d74e98db3c
Merge pull request #55218 from tchaikov/debian/postinst
debian/*.postinst: add adduser as a dependency and specify --home when adduser 

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Adam King adking@redhat.com
2024-02-20 08:10:07 -08:00
luo rixin
450f269235 debian/control: add new dependencies protobuf for crismon
As PR https://github.com/ceph/ceph/pull/55444 update seastar version,
the new seastar imports new dependencies protobuf.

Fixes: https://tracker.ceph.com/issues/64420

Signed-off-by: luo rixin <luorixin@huawei.com>
2024-02-19 17:15:58 +08:00
Shinya Hayashi
a53c0651fb debian: add ceph-exporter package
It is hard for Debian/Ubuntu users to use ceph-exporter
because it is not included in any deb packages.

This commit adds a new deb package for ceph-exporter.

Fixes: https://tracker.ceph.com/issues/64095
Signed-off-by: Shinya Hayashi <shinya-hayashi@cybozu.co.jp>
2024-02-13 01:48:22 +00:00
Kefu Chai
2a30ddccd0 debian/control: add adduser to Depends of cephadm and ceph-common
in `debian/ceph-common.postinst` and `debian/cephadm.postinst`, we
use `adduser --system` to create the system user when configuring
the corresponding package.

before this change, the dependency is not listed in the runtime
`Depends` section of ceph-common and cephadm.

in this change, the dependency is added. this is also suggested
by Securing Debian Manual, see
https://www.debian.org/doc/manuals/securing-debian-manual/bpp-lower-privs.en.html

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-02-03 22:36:50 +08:00
Nizamudeen A
35e40c4e8f mgr/dashboard: adapt to new nvmeof APIs
Fixes: https://tracker.ceph.com/issues/64201
Signed-off-by: Nizamudeen A <nia@redhat.com>
2024-01-30 15:39:35 +05:30
Daniel Persson
c1ea66fe12 mgr/dashboard: Changes suggested after review by @epuertat.
Move the JWT requirement to the test requirements file. Also remove JWT from ceph specification and debian build.

Signed-off-by: Daniel Persson <mailto.woden@gmail.com>
2023-12-13 10:43:01 +01:00
Yuval Lifshitz
46500cace6 rgw/test/lua: add lua integration tests suite
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>

Fixes: https://tracker.ceph.com/issues/63616
2023-11-23 16:36:28 +00:00