-
Notifications
You must be signed in to change notification settings - Fork 14.6k
[libcxx] Update testing documentation about CI container images. #149192
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
cmtice
wants to merge
8
commits into
llvm:main
Choose a base branch
from
cmtice:libc++-documentation
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
a15e371
[libcxx] Update testing documentation about CI container images.
cmtice 757baec
Fix table format.
cmtice 18f213f
More table format fixes.
cmtice bf0596a
Fix typos and other wording issues.
cmtice f37621e
Merge remote-tracking branch 'origin/main' into libc++-documentation
cmtice f0651fb
Move documentation from TestingLibcxx.rst to Contributing.rst
cmtice 0677c57
Update text to address reviewer comments.
cmtice 8d68e8c
Update text as suggested by reviewer.
cmtice File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -174,10 +174,11 @@ Pre-commit CI | |||||
Introduction | ||||||
------------ | ||||||
|
||||||
Unlike most parts of the LLVM project, libc++ uses a pre-commit CI [#]_. This | ||||||
CI is hosted on `Buildkite <https://buildkite.com/llvm-project/libcxx-ci>`__ and | ||||||
the build results are visible in the review on GitHub. Please make sure | ||||||
the CI is green before committing a patch. | ||||||
Unlike most parts of the LLVM project, libc++ uses a pre-commit CI [#]_. Some of | ||||||
this CI is hosted on `Buildkite<https://buildkite.com/llvm-project/libcxx-ci>`__, | ||||||
but some has migrated to the LLVM CI infrastructure. The build results are | ||||||
visible in the review on GitHub. Please make sure the CI is green before | ||||||
committing a patch. | ||||||
|
||||||
The CI tests libc++ for all :ref:`supported platforms <SupportedPlatforms>`. | ||||||
The build is started for every commit added to a Pull Request. A complete CI | ||||||
|
@@ -246,12 +247,15 @@ Below is a short description of the most interesting CI builds [#]_: | |||||
Infrastructure | ||||||
-------------- | ||||||
|
||||||
All files of the CI infrastructure are in the directory ``libcxx/utils/ci``. | ||||||
Note that quite a bit of this infrastructure is heavily Linux focused. This is | ||||||
the platform used by most of libc++'s Buildkite runners and developers. | ||||||
The files for the CI infrastructure are split between the llvm-project | ||||||
and the llvm-zorg repositories. All files of the CI infrastructure in | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
One space after a period, throughout! |
||||||
the llvm-project are in the directory ``libcxx/utils/ci``. Note that | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
quite a bit of this infrastructure is heavily Linux focused. This is | ||||||
the platform used by most of libc++'s Buildkite runners and | ||||||
developers. | ||||||
|
||||||
Dockerfile | ||||||
~~~~~~~~~~ | ||||||
Dockerfile/Container Images | ||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||||||
|
||||||
Contains the Docker image for the Ubuntu CI. Because the same Docker image is | ||||||
used for the ``main`` and ``release`` branch, it should contain no hard-coded | ||||||
|
@@ -261,6 +265,71 @@ GCC, and CMake. | |||||
.. note:: This image is pulled from Docker hub and not rebuild when changing | ||||||
the Dockerfile. | ||||||
|
||||||
Updating the CI testing container images | ||||||
---------------------------------------- | ||||||
|
||||||
The libcxx linux premerge testing can run on one of three sets of runner | ||||||
groups. The three runner group names are "llvm-premerge-libcxx-runners", | ||||||
"llvm-premerge-libcxx-release-runners" and "llvm-premerge-libcxx-next-runners". | ||||||
Which runner set to use is controlled by the contents of | ||||||
https://github.com/llvm/llvm-project/blob/main/.github/workflows/libcxx-build-and-test.yaml. | ||||||
By default, it uses "llvm-premerge-libcxx-runners". To switch to one of the | ||||||
other runner sets, just replace all uses of "llvm-premerge-libcxx-runners" in | ||||||
the yaml file with the desired runner set. | ||||||
|
||||||
Which container image is used by these three runner sets is controlled | ||||||
and set by the variable values in | ||||||
https://github.com/llvm/llvm-zorg/blob/main/premerge/premerge_resources/variables.tf. | ||||||
The table below shows the variable names and | ||||||
the runner sets to which they correspond. To see their values, follow the | ||||||
link above (to variables.tf in llvm-zorg). | ||||||
|
||||||
+------------------------------------+---------------------------+ | ||||||
|Runner Set |Variable | | ||||||
+====================================+===========================+ | ||||||
|llvm-premerge-libcxx-runners |libcxx_runner_image | | ||||||
+------------------------------------+---------------------------+ | ||||||
|llvm-premerge-libcxx-release-runners|libcxx_release_runner_image| | ||||||
+------------------------------------+---------------------------+ | ||||||
|llvm-premerge-libcxx-next-runners |libcxx_next_runner_image | | ||||||
+------------------------------------+---------------------------+ | ||||||
|
||||||
|
||||||
When updating the container image you can either update just the | ||||||
runner binary (the part the connects to Github), or you can update | ||||||
everything (tools, etc.). Whether to update just the runner or to update | ||||||
everything is controlled by the value of ``ACTIONS_BASE_IMAGE``, under | ||||||
``actions-builder`` in ``libcxx/utils/ci/docker-compose.yml``. | ||||||
|
||||||
To update just the runner binary, change the value of ``ACTIONS_BASE_IMAGE`` | ||||||
to be a modified version of one of the libcxx runner variable images from | ||||||
https://github.com/llvm/llvm-zorg/blob/main/premerge/premerge_resources/variables.tf, | ||||||
as follows: Find the libcxx runner image name you want to use from the | ||||||
variables.tf file. The name will be something like | ||||||
``ghcr.io/llvm/libcxx-linux-builder:<some-commit-SHA>``. Replace | ||||||
``libcxx-linux-builder`` with ``libcxx-linux-builder-base``. Use this new image | ||||||
name as the value you assign to ``ACTIONS_BASE_IMAGE``. | ||||||
|
||||||
To update the entire container image, set the value of ``ACTIONS_BASE_IMAGE`` | ||||||
to ``builder-base``. If the value is already ``builder-base`` (there | ||||||
have been no just-the-runner updates since the last complete update), then you | ||||||
need to find the line containing ``RUN echo "Last forced update executed on`` | ||||||
in ``libcxx/utils/ci/Dockerfile`` and update the date to be the current date. | ||||||
|
||||||
Once you have created and merged a PR with those changes, a new image | ||||||
will be created, and a link to it can be found at | ||||||
https://github.com/llvm/llvm-project/pkgs/container/libcxx-linux-builder., | ||||||
where the actual image name should be | ||||||
``ghcr.io/llvm/libcxx-linux-builder:<SHA-of-committed-change-from-PR>``. | ||||||
|
||||||
Lastly you need to create a PR in the llvm-zorg repository, | ||||||
updating the the value of the appropriate libcxx runner variable in | ||||||
the variables.tf file mentioned above to the name of your newly created | ||||||
image (see above paragraph about finding the image name). Once that change | ||||||
has been merged, an LLVM premerge maintainer (a Google employee) must use | ||||||
terraform to apply the change to the running GKE cluster. | ||||||
|
||||||
|
||||||
run-buildbot-container | ||||||
~~~~~~~~~~~~~~~~~~~~~~ | ||||||
|
||||||
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.