gian/ceph-mirror
1
0
Fork 0
mirror of https://github.com/ceph/ceph synced 2026-08-02 07:03:18 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
163,538 Commits 389 Branches 411 Tags 2.4 GiB
main
Commit Graph

3 Commits

Author SHA1 Message Date
Kefu Chai
91de0e1933 cmake: migrate Python module installation from setup.py to pip 
Replace 'setup.py install' with 'pip install --use-pep517' to fix
Cython compilation failures and eliminate deprecation warnings.

Problem Statement:
The build process for Cython modules involves preprocessing .pyx files
(e.g., generating rbd_processed.pyx from rbd.pyx) and then cythonizing
with specific compiler_directives. The previous approach using separate
'setup.py build' and 'setup.py install' commands caused this failure:

```
Error compiling Cython file:
------------------------------------------------------------
...
        """
        name = cstr(name, 'name')
        cdef:
            rados_ioctx_t _ioctx = convert_ioctx(ioctx)
            char *_name = name
            librbd_progress_fn_t _prog_cb = &no_op_progress_callback
                                            ^
------------------------------------------------------------

rbd_processed.pyx:781:44: Cannot assign type 'int (*)(uint64_t, uint64_t, void *) except? -1' to 'librbd_progress_fn_t'. Exception values are incompatible. Suggest adding 'noexcept' to type 'int (uint64_t, uint64_t, void *) except? -1'.
```

This occurs because:
1. 'setup.py build build_ext' successfully preprocesses and cythonizes
   with compiler_directives from setup.py's cythonize() call
2. 'setup.py install' internally triggers a rebuild that:
   - Regenerates the preprocessed .pyx files
   - Re-runs cythonize() through Cython.Distutils.build_ext
   - Does NOT apply the compiler_directives from setup.py
   - Fails on the regenerated files missing required directives

New Options Explained:

`--use-pep517`:
  Addresses deprecation warning:
  ```
    DEPRECATION: Building 'rados' using the legacy setup.py bdist_wheel
    mechanism, which will be removed in a future version. pip 25.3 will
    enforce this behaviour change.
  ```
  Uses the modern PEP 517 build backend which:
  - Performs a single build pass with all compiler_directives applied
  - Prevents the implicit rebuild that caused CompileError
  - Future-proofs against pip 25.3+ which will require this

`--no-build-isolation`:
  Ensures that environment variables set by CMake are respected:
  - CC, LDSHARED (compiler toolchain)
  - CPPFLAGS, LDFLAGS (compilation flags)
  - CYTHON_BUILD_DIR, CEPH_LIBDIR (build paths)

  Without this flag, pip would create an isolated build environment
  that ignores these critical build settings.

`--no-deps`:
  Prevents pip from attempting to install Python dependencies listed
  in setup.py's install_requires. All dependencies are managed by
  CMake and the distribution's package manager, not pip.

`--ignore-installed`:
  Addresses installation error when DESTDIR is set:
  ```
    ERROR: Could not install packages due to an OSError: [Errno 13]
    Permission denied: '/usr/lib/python3/dist-packages/rados-2.0.0.egg-info'

    OSError: [Errno 18] Invalid cross-device link:
    '/usr/lib/python3/dist-packages/rados-2.0.0.egg-info' -> '/tmp/pip-uninstall-...'
  ```
  This error occurs because pip detects an existing system installation
  and tries to uninstall it before installing to DESTDIR. With
  --ignore-installed, pip skips the uninstall step and directly installs
  to the DESTDIR staging directory, which is the correct behavior for
  packaging.

Removed Options:

`--install-layout=deb`:
  This Debian-specific patch to 'setup.py install' is no longer needed.
  Modern pip automatically detects the distribution and uses the correct
  layout (dist-packages on Debian, site-packages on RPM distros).

`--single-version-externally-managed`:
  This option was specific to 'setup.py install' to prevent egg
  installation. With pip, this is handled automatically.

`--record /dev/null`:
  No longer needed as pip manages installation records internally.

`egg_info --egg-base`:
  Not needed with pip as metadata is generated automatically during
  the build process.

Not added option:
`--root-user-action=ignore`: not added
  In this change, we installing a python module using pip with
  `fakeroot` before packaging it. But pip warned:
  ```
  Error: WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behavior with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
  ```
  But we use fakeroot on purpose, this option could have been added  to
  silence this warning. But it is not available in all supported pip
  versions. see
  2e1112a814

New environmental variable:
`DEB_PYTHON_INSTALL_LAYOUT=deb` is conditionally applied when packaging
for debian-derivative distributions. As pip does not support
`--install-layout` option. Since debian patches pip so it installs Python
modules into /usr/local/lib instead of /usr/lib where debian dh_install
helper looks for the content to be packaged, so we have to enforce the
debian layout using the environmental variable.

Working Directory Change:

Changed from `CMAKE_CURRENT_SOURCE_DIR` to `CMAKE_CURRENT_BINARY_DIR` to
keep pip's temporary files and logs in the build directory rather than
polluting the source tree.

Additional Dependencies:

Since the build process uses pip and creates a wheel distribution,
we need to add `pip` and `wheel` Python modules as build dependencies.

Python moduels packaging:

- with `--use-pep517`, pip creates .dist-info directoires as per PEP-517
instead of .egg-info, so we need to package the new metadata directory.

Future Improvements

We considered implementing a custom `build_templates` command or using
setuptools' `sub_commands` mechanism to avoid regenerating `*_processed.pyx`
files on every build (tracking dependencies via file modification times or
hash-based checks). However, to keep `setup.py` simple and maintainable,
we've deferred this optimization for future work. The current solution
using `pip install --use-pep517` ensures correct builds without additional
complexity.

This solution works correctly for both Debian and RPM packaging workflows,
both of which use DESTDIR-based staged installations.

Fixes: 719b749846
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
 2026-01-29 08:45:50 +08:00
James Page
1341de99a0 debian: wrap-and-sort all files 
'wrap-and-sort -a -t' debian folder to apply consistent
ordering/formatting to all files.

This makes synchronization of debian package changes
between Ceph packages and Ubuntu packages easier to
manage.

Signed-off-by: James Page <james.page@ubuntu.com>
 2017-07-04 17:38:39 +01:00
Sage Weil
ffc3dd6e9a debian: add python-rgw package 
Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-09 12:22:46 -05:00