Mold handles several linker features differently from ld.bfd:
- --exclude-libs,ALL hides .symver-aliased symbols (e.g. rados_*)
even when the version script lists them as global. Detect Mold
(via USING_MOLD_LINKER) and disable --exclude-libs; version
scripts already control symbol visibility.
- Duplicate symbols across static archives are treated as errors.
Merge crimson-alien-common into crimson-alienstore as a single
archive. For crimson-osd, link crimson-alienstore before
crimson-common with --allow-multiple-definition so the alien
thread's non-crimson definitions win (Mold picks first definition).
- Python/Cython extension builds (via Distutils.cmake) invoked the
system default linker. Pass -fuse-ld= via MOLD_FUSE_LD_FLAG so
extensions link with the configured linker.
- Add rados_* and _rados_* to librados.map global section for Mold
compatibility with .symver aliases.
Co-authored-by: Mark Kogan <mkogan@redhat.com>
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Replace 'setup.py install' with 'pip install --use-pep517' to fix
Cython compilation failures and eliminate deprecation warnings.
Problem Statement:
The build process for Cython modules involves preprocessing .pyx files
(e.g., generating rbd_processed.pyx from rbd.pyx) and then cythonizing
with specific compiler_directives. The previous approach using separate
'setup.py build' and 'setup.py install' commands caused this failure:
```
Error compiling Cython file:
------------------------------------------------------------
...
"""
name = cstr(name, 'name')
cdef:
rados_ioctx_t _ioctx = convert_ioctx(ioctx)
char *_name = name
librbd_progress_fn_t _prog_cb = &no_op_progress_callback
^
------------------------------------------------------------
rbd_processed.pyx:781:44: Cannot assign type 'int (*)(uint64_t, uint64_t, void *) except? -1' to 'librbd_progress_fn_t'. Exception values are incompatible. Suggest adding 'noexcept' to type 'int (uint64_t, uint64_t, void *) except? -1'.
```
This occurs because:
1. 'setup.py build build_ext' successfully preprocesses and cythonizes
with compiler_directives from setup.py's cythonize() call
2. 'setup.py install' internally triggers a rebuild that:
- Regenerates the preprocessed .pyx files
- Re-runs cythonize() through Cython.Distutils.build_ext
- Does NOT apply the compiler_directives from setup.py
- Fails on the regenerated files missing required directives
New Options Explained:
`--use-pep517`:
Addresses deprecation warning:
```
DEPRECATION: Building 'rados' using the legacy setup.py bdist_wheel
mechanism, which will be removed in a future version. pip 25.3 will
enforce this behaviour change.
```
Uses the modern PEP 517 build backend which:
- Performs a single build pass with all compiler_directives applied
- Prevents the implicit rebuild that caused CompileError
- Future-proofs against pip 25.3+ which will require this
`--no-build-isolation`:
Ensures that environment variables set by CMake are respected:
- CC, LDSHARED (compiler toolchain)
- CPPFLAGS, LDFLAGS (compilation flags)
- CYTHON_BUILD_DIR, CEPH_LIBDIR (build paths)
Without this flag, pip would create an isolated build environment
that ignores these critical build settings.
`--no-deps`:
Prevents pip from attempting to install Python dependencies listed
in setup.py's install_requires. All dependencies are managed by
CMake and the distribution's package manager, not pip.
`--ignore-installed`:
Addresses installation error when DESTDIR is set:
```
ERROR: Could not install packages due to an OSError: [Errno 13]
Permission denied: '/usr/lib/python3/dist-packages/rados-2.0.0.egg-info'
OSError: [Errno 18] Invalid cross-device link:
'/usr/lib/python3/dist-packages/rados-2.0.0.egg-info' -> '/tmp/pip-uninstall-...'
```
This error occurs because pip detects an existing system installation
and tries to uninstall it before installing to DESTDIR. With
--ignore-installed, pip skips the uninstall step and directly installs
to the DESTDIR staging directory, which is the correct behavior for
packaging.
Removed Options:
`--install-layout=deb`:
This Debian-specific patch to 'setup.py install' is no longer needed.
Modern pip automatically detects the distribution and uses the correct
layout (dist-packages on Debian, site-packages on RPM distros).
`--single-version-externally-managed`:
This option was specific to 'setup.py install' to prevent egg
installation. With pip, this is handled automatically.
`--record /dev/null`:
No longer needed as pip manages installation records internally.
`egg_info --egg-base`:
Not needed with pip as metadata is generated automatically during
the build process.
Not added option:
`--root-user-action=ignore`: not added
In this change, we installing a python module using pip with
`fakeroot` before packaging it. But pip warned:
```
Error: WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behavior with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
```
But we use fakeroot on purpose, this option could have been added to
silence this warning. But it is not available in all supported pip
versions. see
2e1112a814
New environmental variable:
`DEB_PYTHON_INSTALL_LAYOUT=deb` is conditionally applied when packaging
for debian-derivative distributions. As pip does not support
`--install-layout` option. Since debian patches pip so it installs Python
modules into /usr/local/lib instead of /usr/lib where debian dh_install
helper looks for the content to be packaged, so we have to enforce the
debian layout using the environmental variable.
Working Directory Change:
Changed from `CMAKE_CURRENT_SOURCE_DIR` to `CMAKE_CURRENT_BINARY_DIR` to
keep pip's temporary files and logs in the build directory rather than
polluting the source tree.
Additional Dependencies:
Since the build process uses pip and creates a wheel distribution,
we need to add `pip` and `wheel` Python modules as build dependencies.
Python moduels packaging:
- with `--use-pep517`, pip creates .dist-info directoires as per PEP-517
instead of .egg-info, so we need to package the new metadata directory.
Future Improvements
We considered implementing a custom `build_templates` command or using
setuptools' `sub_commands` mechanism to avoid regenerating `*_processed.pyx`
files on every build (tracking dependencies via file modification times or
hash-based checks). However, to keep `setup.py` simple and maintainable,
we've deferred this optimization for future work. The current solution
using `pip install --use-pep517` ensures correct builds without additional
complexity.
This solution works correctly for both Debian and RPM packaging workflows,
both of which use DESTDIR-based staged installations.
Fixes: 719b749846
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
The distutils_install_cython_module() function was using ${PY_LDFLAGS}
without defining it, causing the linker to fail with:
/opt/rh/gcc-toolset-13/root/usr/libexec/gcc/x86_64-redhat-linux/13/ld:
cannot find -lrados: No such file or directory
This bug was introduced in commit d22734f6cb which changed:
set(ENV{LDFLAGS} "-L${CMAKE_LIBRARY_OUTPUT_DIRECTORY}")
to:
set(ENV{LDFLAGS} "${PY_LDFLAGS}")
However, PY_LDFLAGS was only defined in distutils_add_cython_module(),
not in distutils_install_cython_module(). This meant that during the
install phase, LDFLAGS was set to an empty string, and the linker
couldn't find librados.so and other Ceph libraries in the build
directory.
The bug was exposed by commit 719b749846
which changed rados.pxd to use C preprocessor conditionals (#ifdef
BUILD_DOC) instead of Cython's compile-time IF statements. This meant
the build now required proper linking during the install phase.
Fix by defining PY_LDFLAGS in distutils_install_cython_module():
set(PY_LDFLAGS "${CMAKE_SHARED_LINKER_FLAGS} -L${CMAKE_LIBRARY_OUTPUT_DIRECTORY}")
This keeps CMAKE_SHARED_LINKER_FLAGS as a space-separated string and
appends the library directory flag, avoiding issues with semicolon
conversion.
Fixes: d22734f6cb
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
if sanitizers are enabled, we have to populate the required link flags
to python extensions's building workflow. otherwise ld would fail to
link like:
```
/usr/bin/ld: /home/jenkins-build/build/workspace/ceph-pull-requests/build/lib/libceph-common.so.2: undefined reference to `__asan_stack_free_10'
```
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
we are facing following FTBFS on jammy + GCC-11.2 + Cython 0.29 +
CMake 3.22:
creating /home/jenkins-build/build/workspace/ceph-api/build/lib/cython_modules/temp.linux-x86_64-3.10/home/jenkins-build/build/workspace/ceph-api/build/src/pybind/cephfs
compile options: '-I/usr/include/python3.10 -I/usr/include/python3.10 -c'
extra options: '-Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -iquote/home/jenkins-build/build/workspace/ceph-api/src/include -w -Dvoid0=dead_function(void) -D__Pyx_check_single_interpreter(ARG)=ARG ## 0 -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2'
cc: /home/jenkins-build/build/workspace/ceph-api/build/src/pybind/cephfs/cephfs.c
cc: warning: ##: linker input file unused because linking not done
cc: error: ##: linker input file not found: No such file or directory
cc: warning: 0: linker input file unused because linking not done
cc: error: 0: linker input file not found: No such file or directory
it seems cython is not able to escape the space in the "extra options"
anymore, so the "##" and "0" are considered as object files passed to
compiler in addition to cephfs.c.
in this change the spaces are removed to help cython to make the right
decision.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
* CMakeLists.txt:
always pass "EXACT" to find_package(Python3).
because per cmake document, "EXACT" only takes effect when
<Package>_FIND_VERSION_COUNT is greater than 1, where <Package>
is "Python3". see also cmake/modules/FindPython/Support.cmake
* cmake/modules/AddCephTest.cmake:
drop redundant find_package(Python3) calls. since Python3 is
a mandatory requirement for building Ceph, we only need a
single call of find_package(Python3..) in the top of the source
tree. the only possible case to repeat it is to ensure that we
have the correct version of Python3 used in following CMake
script. but there is no need to repeat it if we just want to
ensure that we have a python3 interpretor in place.
* cmake/modules/Distutils.cmake:
always pass "EXACT" to find_package(Python3).
we should always pass EXACT to find_package() when finding python3,
this is a follow-up of e2babdfae8
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
if another python3 with higher version is found by
find_package(Python3), the cmake's install script would just
install the python modules/extensions into that python3's
dist-package directory, and the packaging script would fail
to find these artifacts when trying to package them.
so we need to ensure that the install directories for python
modeules/extensions are always "versioned" with WITH_PYTHON3
cmake option.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
to address following warning from python 3.9:
<string>:1: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
<string>:1: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
this change make it more explicit how we override CC, CFLAGS and
CPPFLAGS:
* CC only for the compiler
* CFLAGS only for the flags consumed by C compiler
* CPPFLAGS only used for the preprocessor, it is used by both C and C++
compilers.
this change does not address any specific test failure. it just
improves the correctness and readability.
Signed-off-by: Kefu Chai <kchai@redhat.com>
to silence the warning like:
/home/jenkins-build/build/workspace/ceph-pull-requests/build/src/pybind/rados/rados.c: In function '__pyx_pymod_exec_rados':
/home/jenkins-build/build/workspace/ceph-pull-requests/build/src/pybind/rados/rados.c:78903:30: note: variable tracking size limit exceeded with '-fvar-tracking-assignments', retrying without
78903 | static CYTHON_SMALL_CODE int __pyx_pymod_exec_rados(PyObject *__pyx_pyinit_module)
| ^~~~~~~~~~~~~~~~~~~~~~
cc: warning: /usr/bin/cc: linker input file unused because linking not done
Signed-off-by: Kefu Chai <kchai@redhat.com>
it's reported that on Alphine, "make install" installs
`ceph-volume-systemd` into `/usr/bin` instead of `/usr/sbin`, and this
breaks `systemd/ceph-volume@.service`. so in this change, we always
install this script into `/usr/sbin`, even if `DESTDIR` is not defined.
Signed-off-by: Kefu Chai <kchai@redhat.com>
instead of appending compile flags to CMAKE_C_FLAGS, use
add_compile_options(), as COMPILE_OPTIONS is a list, it'd simpler to
append options to it and to access it in a structured way.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* drop WITH_PYTHON2 option
* print warning message if WITH_PYTHON3 is disabled
* drop MGR_PYTHON_VERSION option, as we don't support use different
python version for python binding and ceph-mgr embedded python
interpreter anymore. as before switching to python3-only build,
we can build python3 and python2 python bindings, and ceph-mgr
can use either of them. but after switching to python3-only
build, ceph-mgr has to use whatever python version used to
build python binding.
* move WITH_PYTHON3 option to $top_srcdir/CMakeLists.txt, as ceph-mgr
and python binding will share this option.
* hardware ${PYTHON_VERSION} to 3
* hardware ${Python${PYTHON_VERSION}_VERSION_MAJOR} to 3
* only build boost library with python3
* s/Python_EXECUTABLE/Python3_EXECUTABLE/
* update the build scripts and packagings accordingly
* rename all cython${PYTHON_VERSION}_* targets to cython_*
* update distutils_install_module() so it does not take python_version
parameter anymore.
Signed-off-by: Kefu Chai <kchai@redhat.com>
if `distutils_install_module("foo" ...)` is called mutiple times with
different python version, `foo-clone` will be added multiple times as a
custom target. which is not allowed:
add_custom_target cannot create target "foo-clone" because another
target with the same name already exists.
Signed-off-by: Kefu Chai <kchai@redhat.com>
allow `distutils_*()` functions to choose the python according to this
parameter, instead of implicitly using the variable of ${PYTHON_VERSION}
inherited from the caller.
Signed-off-by: Kefu Chai <kchai@redhat.com>
use the ones shiped from the latest cmake. which
* enables us to find the recent python intepreter and development files,
* find intepreter and development in a single `find_pacakge()` command,
simpler this way and less error prone.
and to accomodate this change:
* all `PYTHON${PYTHON_VERSION}_*` variables are renamed to
`Python${PYTHON_VERSION}_*` if we use `find_package(Python2...)` or
`find_package(Python3...)` to find python2 or python3 instead of using
`find_package(Python...)`.
* use "2" explicitly when using python2, as `Python_*` variables are not
defined anymore
* when compiling python support of ceph-mgr, continue using `Python_*`
variables. because we find the python interpreter and development
files using `find_pacakge(Python...)` for ceph-mgr.
Signed-off-by: Kefu Chai <kchai@redhat.com>
use the combination of add_custom_command() and add_custom_target() to
build cython extensions to avoid rebuilding the cython extension
everytime we run "make"
Signed-off-by: Kefu Chai <kchai@redhat.com>
so add_custom_target() has better idea what is generated. if any target
will use it, it can be used as a dependency.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Commit 3bde34af8a removed cython 0.29's subinterpreter check when
building the various python modules during `make`, but unforunately
they're *rebuilt* during `make install`, with the rebuild overwriting
the original build. The original fix was of course missing from the
install stage...
Fixes: https://tracker.ceph.com/issues/38788
Signed-off-by: Tim Serong <tserong@suse.com>
cython 0.29 introduced a check which prevents multiple python
subinterpreters from loading the same module:
https://github.com/cython/cython/commit/7e27c7c
Unfortunately, this completely breaks ceph-mgr. Until we can
figure out a better long term solution, this commit removes
cython's subinterpreter check, via some careful abuse of the
C preprocessor.
This works because when cython is invoked, it first generates
some C code, then compiles it. We know it's going to generate
C code including:
int __Pyx_check_single_interpreter(void) { ... }
and:
if (__Pyx_check_single_interpreter())
return NULL;
So, we can do the following:
#define void0 dead_function(void)
#define __Pyx_check_single_interpreter(ARG)=ARG ## 0
This replaces the call to __Pyx_check_single_interpreter()
with a literal 0, removing the subinterpreter check.
The void0 dead_function(void) thing is necessary because
the __Pyx_check_single_interpreter() macro also clobbers
that function definition, so we need to make sure it's
replaced with something that works as a function definition.
Fixes: https://tracker.ceph.com/issues/37472
Signed-off-by: Tim Serong <tserong@suse.com>
--single-version-externally-managed is used for packaging python eggs
to deb. so the metadata can be stored in the egg, and can be accessed
by user if necessary, see [1]. but this does not apply to users who
just want to "make install" without packaging Ceph.
the problem is, if we pass --single-version-externally-managed to
setup.py, we need to pass --record or --root also. otherwise setuptools
complains:
error: You must specify --record or --root when building system packages
so in this change, we choose not to pass
"--single-version-externally-managed", unless $DESTDIR is set.
both "ceph.spec.in" and "debian/rules" set $DESTDIR when running "make
install". so we can always check $DESTDIR to tell if current "make
install" is launched by the packaging script or not. this might not be
accurate, but good enough. assuming enduser does not use DESTDIR when
running plain "make install".
--
[1] https://wiki.debian.org/Python/FAQ#How_should_we_package_Python_eggs.3F
Signed-off-by: Kefu Chai <kchai@redhat.com>
in python's distutils.ccompiler, linker_exe is composed using CC instead
of LDFLAGS. the latter only effects how it builds (shared) library.
and put CMAKE_C_FLAGS into the cflags for the compiler for building
python C extensions, it's more consistent this way. more importantly,
if we build with ASan enabled, the canary program, a.k.a. rados_dummy.c,
won't link without proper CFLAGS.
without this change, rados.so fails to build with errors like:
/usr/bin/ld: /var/ssd/ceph/build/lib/librados.so: undefined reference to
`__asan_stack_free_10'
/usr/bin/ld: /var/ssd/ceph/build/lib/librados.so: undefined reference to
`__asan_report_exp_store8'
...
...
clang: error: linker command failed with exit code 1 (use -v to see
invocation)
Link Error: RADOS library not found
make[3]: ***
[src/pybind/rados/CMakeFiles/cython_rados.dir/build.make:57:
src/pybind/rados/CMakeFiles/cython_rados] Error 1
Signed-off-by: Kefu Chai <kchai@redhat.com>
When building with ccache, distcc, and other compiler wrappers (such
as STLFilt):
CC='ccache gcc' CXX='ccache g++' cmake /path/to/ceph
make
python modules fail to compile since distutils try to execute the
wrapper itself without specifying the actual compiler.
Although cmake has a special magic switch for compiling with ccache
(cmake -DWITH_CCACHE=ON) other tools (distcc) are not supported, and
specifying the compiler as
CC=/whatever/compiler/is
used to work for decades, and it's a good idea to keep it working
Signed-off-by: Alexey Sheplyakov <asheplyakov@mirantis.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
otherwise, the default gcc will be used, and the $CMAKE_C_COMPILER
passed to the outer CMakeLists.txt won't kick in. moreover, if the
building script (ceph.spec for instance) could set the $PATH, and
expect that the CMakeLists.txt will use the toolchain executables
in the $PATH to build Ceph, distutils will continue using the default
$CC for linking the python bindings, on UNIX it will be gcc in the
new shell's $PATH, because we are using `install(CODE "... execute_process(
...))` for installing the python bindings. apparently, this is not
expected. because the new shell's $PATH is very likely different
from the one changed by the building script. to address this, we
should always specify the `$LDSHARED` env var explicitly.
also, pass env vars using `ENV{}` instead of the `env` command to
workaround the issue of https://cmake.org/pipermail/cmake/2015-December/062216.html,
because it's not straightforward to set environment variables with
spaces in the them using cmake. and because one cannot use add_custom_target()
in the script mode of cmake. this leave me only limited options to
fix this issue.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Skip nonexistent 'expected' Python packaging files, and add bin/ as
a possible file for symlinking during build (for Python projects with
slightly different forms, currently ceph-volume)
Signed-off-by: Dan Mick <dan.mick@redhat.com>
distutils' install command installs all stuff under $build-base, which
includes the temporary files. and rpm packaging does not like::
Installed (but unpackaged) file(s)
and to support both py2 and py3, we should encode the python version
into the path, otherwise the built files will be overwritten.
Signed-off-by: Kefu Chai <kchai@redhat.com>
the empty .so file does not compile at all, so we need to bypass the
sanity check, if we are building docs. and what sphinx needs is just the
docstrings.
Fixes: http://tracker.ceph.com/issues/16940
Signed-off-by: Kefu Chai <kchai@redhat.com>
it was put into the directory where .pyx is located. it's not expected
in an out-of-tree build.
* {rados,rbd,cephfs}/setup.py: use build_ext from cython if possible, fallback to
the one from setuptools or distutils
* Distutils.cmake: pass --cython-c-in-temp --build-temp and
--cython-include-dirs to "build_ext" command
Signed-off-by: Kefu Chai <kchai@redhat.com>
otherwise they will go to the source tree, and "git ls-files" will list
them as untracked files, which annoy gitbuilder-ceph-tarball*-cmake
gitbuilders. like
+ echo 'error: Added files:'
error: Added files:
+ cat .git/added-files
src/pybind/rados/rados.egg-info/PKG-INFO
...
Signed-off-by: Kefu Chai <kchai@redhat.com>
* Distutils.cmake:
set --prefix=${CMAKE_INSTALL_PREFIX} for python packages installed using
setuptools. it was set to --prefix=/user only when $DESTDIR is set. so
if user installs ceph using -DCMAKE_INSTALL_PREFIX, these python
packages still go to /usr, which is unexpected.
* ceph-disk/CMakeLists.txt:
install script into ${CMAKE_INSTALL_SBINDIR} instead of /usr/sbin
Signed-off-by: Kefu Chai <kchai@redhat.com>
* add keyword "INSTALL_SCRIPT" to distutils_install_module(), so we can
override the install path of ceph-disk script.
* refactor the Distutils.cmake module a little bit, the ${option}
variable out lives the CODE snipplet. so we need to reset it
at the beginning.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* fix CYTHON_ADD_MODULE() macro. because python_add_module() offered by
FindPythonLibs.cmake creates a target with name of ${name}, which conflicts
with existing targets like "rbd" or "rados". so we can not reuse the
name in ${name}.pyx. and instead, we should specify the target name
explicitly.
* add distutils_install_cython_module() function to build and install
cython modules.
* we can split build and install of cython module, but the install phase
always tries to build the module. so keep it this way. will look at it
later on.
* move the variables initializations into the Distutils.cmake module.
Signed-off-by: Kefu Chai <kchai@redhat.com>
