apt-mirror.front.sepia.ceph.com has happened to always work because we set up CNAMEs to gitbuilder.ceph.com.
That host is making its way to a new home upstate (literally and figuratively) so we'll get rid of the front subdomain since it's publicly accessible anyway and add TLS while we're at it.
Signed-off-by: David Galloway <david.galloway@ibm.com>
With ceph#66229 merged, crimson is now enabled by default for all the builds. So if we're trying to build with crimson disabled, the bcond must be reversed to install the relavant dependencies.
Signed-off-by: T K Chandra Hasan <t.k.chandra.hasan@ibm.com>
function ensure_decent_gcc_on_rh() is supposed to log a warning when it
finds the default gcc version is older than the gcc toolset version
installed by ceph.spec.in
> Your GCC is too old. Please run following command to add DTS to your environment:
however, this function is only called if the dts_ver variable is
initialized, so set it to 13 for centos/rocky 9 and later to match
ceph.spec.in
Signed-off-by: Casey Bodley <cbodley@redhat.com>
This code used to run in a pbuilder hook (because it needed to run
inside the build environment chroot). When building in a container,
you also want the right compiler installed.
This is necessary at least to build reef on ubuntu focal.
Fixes: https://tracker.ceph.com/issues/73585
Signed-off-by: Dan Mick <dan.mick@redhat.com>
Previously, the FOR_MAKE_CHECK variable could only enable installing
extra (test) dependencies when install-deps.sh was used and it was
ignored if `tty -s` exited true. This change allows FOR_MAKE_CHECK to
take precedence over the tty check and to specify one of true, 1, yes to
enable extra "for make check" deps or false, 0, no to explicitly disable
the extra deps.
Based-on-work-by: Dan Mick <dan.mick@redhat.com>
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Clang relies on the system's libstdc++ headers for standard C++
features. On Ubuntu Jammy, gcc-11 is the default and it's libstdc++ is
missing some of the newer features supported by C++20.
This causes build failures as certain standard headers are incomplete.
To fix this, we install gcc-12 explicitly so that Clang can pick up the
newer libstdc++ headers.
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
Add a new dnf/yum repository hosted in the ceph lab infra for providing
the last few dependencies missing from other el10 repos.
Hopefully we can remove this soon but it serves as a stopgap as we work
on getting el10 builds working in the ceph CI infra and tested.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Rocky Linux 10 has a CRB repo that must be enabled for a number of
Ceph's dependencies to be installed via dnf.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
While the function is pretty simple and could be copy-pasted I
prefer to extract things into functions to indicate that the
logic is used/repeated elsewhere to ward off making changes to
one copy vs the other.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
If we add a new Boost component, the install-deps.sh script will not install it
because other Boost libraries are installed with the desired version. Just
unconditionally install. The package manager is smart enough to skip already
installed packages.
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
The centos 10 stream distribution is like centos 9 and needs the crb
repository enabled to even think about building ceph. Note that I made
it a separate block in case more stuff needs to be added for it later.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
This is actually a follow up patch for the PR #55054 and
commit 03e11acca0
and makes clean up for suse based distros.
Fixes: 03e11acca0
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@gmail.com>
Since ccache 4.0, ccache use $XDG_CACHE_HOME/ccache to keep compile cache
if XDG_CACHE_HOME is set. In this case $XDG_CACHE_HOME is overwrite,
ccache will use $XDG_CACHE_HOME/ccache(ccache will create the dir if not exsit) to
store compile cache, but $XDG_CACHE_HOME will be removed next round running,
leading to ccache contests are always removed. So save and restore user's XDG_CACHE_HOME.
Fixes: https://tracker.ceph.com/issues/65175
Signed-off-by: luo rixin <luorixin@huawei.com>
As `apt-get remove ceph-libboost*` show the error and cause the make check failed.
`E: The package ceph-libboost1.79-dev needs to be reinstalled, but I can't find an archive for it.`
The ceph-libboost* packages may be not removed, so use `dpkg` to force remove ceph-libboost*.
Fixes: https://tracker.ceph.com/issues/64032
Signed-off-by: luo rixin <luorixin@huawei.com>
Here, we extract `clean_boost_on_ubuntu()` and call it before other
installs on Debian distributions so that if we install a system boost,
a potentially newer `ceph-libboost` won't get in the way.
As the sources.list.d being removed in the original cleanup code isn't
the one we're currently installing in the install code, add a removal
for the currently used source, then do apt-update so packages from the
removed source are no longer included as available.
Two subsidiary dev packages from conflicting boost libraries can be
installed, but it leaves apt in an inconsistent state. To clean this
up, add `--fix-missing` to the removal line and call
`clean_boost_on_ubuntu()` before other uses of apt.
Fixes: https://tracker.ceph.com/issues/62097
Signed-off-by: Adam Emerson <aemerson@redhat.com>
This was a workaround that was introduced long time ago. This will be
something that could be deprectaed at some point [1]. And its preventing some of the dependencies to be
downloaded or stored into the wheelhouse. Deps like jsonschema, parse,
mypy, cryptography etc.
[1] https://pip.pypa.io/en/latest/user_guide/#deprecation-timeline
Fixes: https://tracker.ceph.com/issues/62082
Signed-off-by: Nizamudeen A <nia@redhat.com>
build scripts: various cleanups and minor changes preparing for containerized build and test
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Copy the logic from run-make.sh into install-deps.sh so that we can later
remove it from run-make.sh. It helps prevent breakage when apt-get is
interrupted.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Instead of requiring other scripts to install packages independently,
teach install-deps.sh to install additional packages from the variable
INSTALL_EXTRA_PKGS. Now, other scripts should just set
INSTALL_EXTRA_PKGS and call install-deps.sh.
In particular, this fixes an issue installing packages in a clean (ex.
container) system that doesn't yet have repositories set up. Since this
task is performed by install-deps.sh already we avoid a chicken-and-egg
issue (or doing redundant work of setting up repos) in other scripts.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Previously, the main part (top level body) of the script started and
then some function definitions occurred and then the main part of the
script resumed after that. I, and others, find this confusing so this
change moves the function definitions to occur before the main body of
the install-deps.sh script.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
More specifically, if we get e.g. a 504 while attempting to download a
.list file, fail the build and log the status code.
Signed-off-by: Zack Cerza <zack@redhat.com>
The en_US.UTF-8 locale is not always going to present on all systems.
Specifically, setting en_US.UTF-8 causes numerous warnings when running
this script on a CentOS Stream 8 container. Rather than install
additional packages to suppress the warning, use the C.UTF-8 locale.
Additionally, try to update the comment explaining the line to something
based on the commit history rather than the confusing term "vulnerable".
Signed-off-by: John Mulligan <jmulligan@redhat.com>
The majority of the install-deps.sh file was already using spaces to
indent, convert the remaining hard tabs to 8 spaces.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
The majority of the lines in this shell script uses spaces to indent,
but I *think* this modeline tells emacs to indent with tabs. In a very
near future change I'm going to make all indents use spaces. So we first
remove the emacs modeline that tells it to use the opposite.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
When run on focal and bionic, install-deps ends early
due to this extra debug message that was added to the
end of `ensure_decent_gcc_on_ubuntu`. The debug message
prints when the script is run in a jenkins environment.
When the script is not run in a jenkins environment, the
value returned there is "false" or "0", which acts as
an early return. This stops the script from completing.
We can remove this line, as `ensure_decent_gcc_on_ubuntu`
is only called for focal and bionic, and most of the jenkins
nodes are running jammy. Also, there is a debug message at the
beginning of the function that should suffice.
Fixes: https://tracker.ceph.com/issues/57466
Signed-off-by: Laura Flores <lflores@redhat.com>
