From 589e5ba14a2a4357625b43234a91b4e6cf2e3a63 Mon Sep 17 00:00:00 2001 From: Louis Dionne Date: Tue, 9 Dec 2025 08:45:23 -0500 Subject: [PATCH 1/2] [libc++] Store the premerge runner images in the monorepo We need one canonical place to store the image used by the various sets of libc++ CI runners. This is needed so that our run-buildbot-container script can stay up-to-date, and for the pre-merge infrastructure to stay up-to-date. Previously, the images used by the premerge infrastructure were stored in llvm-zorg, which makes it less discoverable and more complicated to update and keep synchronized. --- libcxx/docs/Contributing.rst | 42 +++++-------------- .../utils/ci/images/libcxx_next_runners.txt | 1 + .../ci/images/libcxx_release_runners.txt | 1 + libcxx/utils/ci/images/libcxx_runners.txt | 1 + libcxx/utils/ci/run-buildbot-container | 5 ++- 5 files changed, 17 insertions(+), 33 deletions(-) create mode 100644 libcxx/utils/ci/images/libcxx_next_runners.txt create mode 100644 libcxx/utils/ci/images/libcxx_release_runners.txt create mode 100644 libcxx/utils/ci/images/libcxx_runners.txt diff --git a/libcxx/docs/Contributing.rst b/libcxx/docs/Contributing.rst index e660daeba7e5b..84bbe1478c512 100644 --- a/libcxx/docs/Contributing.rst +++ b/libcxx/docs/Contributing.rst @@ -271,31 +271,18 @@ Updating the CI testing container images The libcxx linux premerge testing can run on one of three sets of runner groups. The three runner group names are ``llvm-premerge-libcxx-runners``, ``llvm-premerge-libcxx-release-runners`` and ``llvm-premerge-libcxx-next-runners``. -The runner set to use is controlled by the contents of +The runner set currently in use is controlled by the contents of https://github.com/llvm/llvm-project/blob/main/.github/workflows/libcxx-build-and-test.yaml. By default, it uses ``llvm-premerge-libcxx-runners``. To switch to one of the other runner sets, just replace all uses of ``llvm-premerge-libcxx-runners`` in the yaml file with the desired runner set. -Which container image is used by these three runner sets is controlled -and set by the variable values in -https://github.com/llvm/llvm-zorg/blob/main/premerge/premerge_resources/variables.tf. -The table below shows the variable names and -the runner sets to which they correspond. To see their values, follow the -link above (to ``variables.tf`` in llvm-zorg). - -+------------------------------------+---------------------------+ -|Runner Set |Variable | -+====================================+===========================+ -|llvm-premerge-libcxx-runners |libcxx_runner_image | -+------------------------------------+---------------------------+ -|llvm-premerge-libcxx-release-runners|libcxx_release_runner_image| -+------------------------------------+---------------------------+ -|llvm-premerge-libcxx-next-runners |libcxx_next_runner_image | -+------------------------------------+---------------------------+ - - -When updating the container image you can either update just the runner binary (the part +The container image used by these three runner sets is controlled by the contents +of the corresponding text files in ``libcxx/utils/ci/images``. The content of these +files is read by the `Terraform configuration in llvm-zorg +`__. + +When updating the container image, you can either update just the runner binary (the part that connects to Github), or you can update everything (tools, etc.). To update the runner binary, bump the value of ``GITHUB_RUNNER_VERSION`` in ``libcxx/utils/ci/docker/docker-compose.yml``. To update all of the tools, bump ``BASE_IMAGE_VERSION`` to a newer version of the ``libcxx-linux-builder-base`` @@ -304,16 +291,9 @@ image. You can see all versions of that image at https://github.com/llvm/llvm-pr On push to ``main``, a new version of both the ``libcxx-linux-builder`` and the ``libcxx-android-builder`` images will be built and pushed to https://github.com/llvm/llvm-project/packages. -You can then update the image used by the actual runners by changing the sha associated -to ``libcxx_runner_image``, ``libcxx_release_runner_image`` or ``libcxx_next_runner_image`` -in `the Terraform configuration file `_. -To do so, you will need to create a PR in the llvm-zorg repository and wait for it to be -merged. Once that change has been merged, an LLVM premerge maintainer (a Google employee) -must use terraform to apply the change to the running GKE cluster. - -.. note:: When you update the ``libcxx_runner_image``, also make sure to update the - ``libcxx/utils/ci/run-buildbot-container`` script to contain the new image. - +You can then update the image used by the actual runners by changing the image encoded in +``libcxx/utils/ci/images`` and asking an LLVM premerge maintainer (a Google employee) to +actually deploy the changes to the GKE cluster via Terraform. Monitoring premerge testing performance ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -336,7 +316,7 @@ your system. run-buildbot ~~~~~~~~~~~~ -Contains the build script executed on Buildkite. This script can be executed +This is the script executed by the CI runners. This script can be executed locally or inside ``run-buildbot-container``. The script must be called with the target to test. For example, ``run-buildbot generic-cxx20`` will build libc++ and test it using C++20. diff --git a/libcxx/utils/ci/images/libcxx_next_runners.txt b/libcxx/utils/ci/images/libcxx_next_runners.txt new file mode 100644 index 0000000000000..eafa0fe5ff6d8 --- /dev/null +++ b/libcxx/utils/ci/images/libcxx_next_runners.txt @@ -0,0 +1 @@ +ghcr.io/llvm/libcxx-linux-builder:55f634cc7f345a8708f2497c0274144ab1eb3d91 \ No newline at end of file diff --git a/libcxx/utils/ci/images/libcxx_release_runners.txt b/libcxx/utils/ci/images/libcxx_release_runners.txt new file mode 100644 index 0000000000000..58f0718c1ce48 --- /dev/null +++ b/libcxx/utils/ci/images/libcxx_release_runners.txt @@ -0,0 +1 @@ +ghcr.io/llvm/libcxx-linux-builder:05b5090e961f6b45ba9144d4d846e0f174d0aedf \ No newline at end of file diff --git a/libcxx/utils/ci/images/libcxx_runners.txt b/libcxx/utils/ci/images/libcxx_runners.txt new file mode 100644 index 0000000000000..4389aaa73866f --- /dev/null +++ b/libcxx/utils/ci/images/libcxx_runners.txt @@ -0,0 +1 @@ +ghcr.io/llvm/libcxx-linux-builder:d6b22a347f813cf4a9832627323a43074f57bbcf \ No newline at end of file diff --git a/libcxx/utils/ci/run-buildbot-container b/libcxx/utils/ci/run-buildbot-container index fa83d1db4f40f..2e5dac92a066b 100755 --- a/libcxx/utils/ci/run-buildbot-container +++ b/libcxx/utils/ci/run-buildbot-container @@ -26,6 +26,7 @@ if [[ ! -d "${MONOREPO_ROOT}/libcxx/utils/ci" ]]; then echo "Was unable to find the root of the LLVM monorepo; are you running from within the monorepo?" exit 1 fi -docker pull ghcr.io/llvm/libcxx-linux-builder:d6b22a347f813cf4a9832627323a43074f57bbcf -docker run -it --volume "${MONOREPO_ROOT}:/llvm" --workdir "/llvm" --cap-add=SYS_PTRACE ghcr.io/llvm/libcxx-linux-builder:d6b22a347f813cf4a9832627323a43074f57bbcf \ +image="$(cat ${MONOREPO_ROOT}/libcxx/utils/ci/images/libcxx_runners.txt)" +docker pull ${image} +docker run -it --volume "${MONOREPO_ROOT}:/llvm" --workdir "/llvm" --cap-add=SYS_PTRACE ${image} \ bash -c 'git config --global --add safe.directory /llvm ; exec bash' From e100270636c47e6f32a00fb8f226efb176dbbec4 Mon Sep 17 00:00:00 2001 From: Louis Dionne Date: Thu, 11 Dec 2025 08:50:57 -0500 Subject: [PATCH 2/2] Bump runner image --- libcxx/utils/ci/images/libcxx_runners.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libcxx/utils/ci/images/libcxx_runners.txt b/libcxx/utils/ci/images/libcxx_runners.txt index 4389aaa73866f..d476700632ff2 100644 --- a/libcxx/utils/ci/images/libcxx_runners.txt +++ b/libcxx/utils/ci/images/libcxx_runners.txt @@ -1 +1 @@ -ghcr.io/llvm/libcxx-linux-builder:d6b22a347f813cf4a9832627323a43074f57bbcf \ No newline at end of file +ghcr.io/llvm/libcxx-linux-builder:36a95a5cb49bb723470c9c967a6bd54d08743834 \ No newline at end of file