Skip to content

Upload CI images to Leafcloud S3 for image syncing #752

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Aug 7, 2025
Merged

Upload CI images to Leafcloud S3 for image syncing #752

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f49193c
Image sync CI to use leafcloud S3
bertiethorpe Aug 5, 2025
a36f490
Use leafcloud s3 config
bertiethorpe Aug 6, 2025
a8777e1
Merge branch 'main' into feat/ci-s3-to-leafcloud
bertiethorpe Aug 6, 2025
931eebe
Update release image workflows and upgrades.md bucket url
bertiethorpe Aug 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/bin/get-s3-image.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ if [ -n "$image_exists" ]; then
else
echo "Image $image_name not found in OpenStack. Getting it from S3."

wget https://object.arcus.openstack.hpc.cam.ac.uk/swift/v1/AUTH_3a06571936a0424bb40bc5c672c4ccb1/$bucket_name/$image_name --progress=dot:giga
wget https://leafcloud.store/swift/v1/AUTH_f39848421b2747148400ad8eeae8d536/$bucket_name/$image_name --progress=dot:giga

echo "Uploading image $image_name to OpenStack..."
openstack image create --file $image_name --disk-format qcow2 $image_name --progress
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release-image.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ jobs:
- uses: actions/checkout@v2

- name: Write s3cmd configuration
run: echo "${{ secrets.ARCUS_S3_CFG }}" > ~/.s3cfg
run: echo "${{ secrets.LEAFCLOUD_S3_CFG }}" > ~/.s3cfg

- name: Install s3cmd
run: |
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/s3-image-sync.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Upload CI-tested images to Arcus S3 and sync clouds
name: Upload CI-tested images to Leafcloud S3 and sync clouds
on:
workflow_dispatch:
push:
Expand All @@ -21,7 +21,7 @@ jobs:

- name: Write s3cmd configuration
run: |
echo "${{ secrets['ARCUS_S3_CFG'] }}" > ~/.s3cfg
echo "${{ secrets['LEAFCLOUD_S3_CFG'] }}" > ~/.s3cfg
shell: bash

- name: Install s3cmd
Expand Down Expand Up @@ -73,7 +73,7 @@ jobs:

- name: Write s3cmd configuration
run: |
echo "${{ secrets['ARCUS_S3_CFG'] }}" > ~/.s3cfg
echo "${{ secrets['LEAFCLOUD_S3_CFG'] }}" > ~/.s3cfg
shell: bash

- name: Install s3cmd and qemu-utils
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/upload-release-image.yml.sample
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ on:
inputs:
image_name:
type: string
description: Image name from: (https://object.arcus.openstack.hpc.cam.ac.uk/swift/v1/AUTH_3a06571936a0424bb40bc5c672c4ccb1/{BUCKET_NAME})
description: Image name from: (https://leafcloud.store/swift/v1/AUTH_f39848421b2747148400ad8eeae8d536/{BUCKET_NAME})
required: true
bucket_name:
type: choice
Expand Down
2 changes: 1 addition & 1 deletion docs/upgrades.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ All other commands should be run on the Ansible deploy host.
1. Identify image(s) from the relevant [Slurm appliance release](https://github.com/stackhpc/ansible-slurm-appliance/releases), and download
using the link on the release plus the image name, e.g. for an image `openhpc-RL9-250708-1547-1494192e`:

wget https://object.arcus.openstack.hpc.cam.ac.uk/swift/v1/AUTH_3a06571936a0424bb40bc5c672c4ccb1/openhpc-images/openhpc-RL9-250708-1547-1494192e
wget https://leafcloud.store/swift/v1/AUTH_f39848421b2747148400ad8eeae8d536/openhpc-images/openhpc-RL9-250708-1547-1494192e

Note that some releases may not include new images. In this case use the image from the latest previous release with new images.

Expand Down