Skip to content

Commit 98f31ef

Browse files
4141done4141done
authored
Build multi-architecture images (#197)
# What Allows us to push multi-architecture builds for `linux/amd64` and `linux/arm64` on merge to master. Fixes: #196 Tests are performed by building the relevant image and tagging it with the image name the tests expect. This follows the original implementation in `test.sh`. This will likely change to an explicit image name specification as we move the test suite in to Javascript but it's being maintained for now in the interest of simplicity. ## Goals * To decouple image build from the test run. This will help as we refactor the test suite for this issue #191. * Not have CI take a long time due to multi-architecture builds for interim pushes to pull requests. * CI should fail fast if any of the image builds or tests fail * Parallelize builds as much as possible. ## Notes * Since the `unprivileged` and `latest-njs` images build off the base `oss` image, there were some strange moves that had to be done to make that work without rebuilding the base image every time. Specifically, we had to set `setup-buildx-action` to run in `docker` driver mode so we can simply `load` the base image. Otherwise we would have had to use a local repository. However, in `docker` mode `upload-artifact` doesn't like the file produced so we have to save the file again. * I wanted to build images once and then run tests against them and push at the end. However, there was not a clean way to get the full multi-architecture images all the way to the push step so I just build them again against all architectures and perform tests in the runner architecture. This saves us from needing conditionals in the test portions.
1 parent 231fa7d commit 98f31ef

File tree

2 files changed

+268
-119
lines changed

2 files changed

+268
-119
lines changed

‎.github/workflows/main.yml‎

Lines changed: 241 additions & 96 deletions
Original file line numberDiff line numberDiff line change
@@ -8,16 +8,54 @@ on:
88
pull_request:
99
branches: [ master ]
1010

11-
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
12-
jobs:
13-
build:
14-
runs-on: ubuntu-latest
1511

16-
if: github.ref != 'refs/heads/master'
12+
env:
13+
CI: true
14+
15+
16+
# Job progression. We make sure that the base image [oss] builds and passes tests before kicking off the other builds
17+
18+
# ┌──────────────────┐ ┌────────────────┐ ┌────────────────┐
19+
# ┌─────────┐ ┌─────────┬────► Build Latest NJS ├────────►Test Latest NJS ├─────►│Push Latest NJS │
20+
# │Build OSS├────►│Test OSS │ └──────────────────┘ └────────────────┘ └────────────────┘
21+
# └─────────┘ └──┬──────┤
22+
# │ │ ┌──────────────────┐ ┌──────────────────┐ ┌─────────────────┐
23+
# │ └────►Build Unprivileged├───────►Test Unprivileged ├────►│Push Unprivileged│
24+
# │ └──────────────────┘ └──────────────────┘ ├────────┬────────┘
25+
# │ ├────────┤
26+
# └──────────────────────────────────────────────────────────────►│Push OSS│
27+
# └────────┘
28+
29+
# As a last step, if we are on the main/master branch, multi-architecture images will be built and pushed to github packages
30+
# and docker hub
31+
32+
jobs:
33+
build-oss-for-test:
34+
runs-on: ubuntu-22.04
1735
steps:
18-
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
19-
- uses: actions/checkout@v2
36+
- uses: actions/checkout@v4
37+
- name: Set up Docker Buildx
38+
uses: docker/setup-buildx-action@v3
39+
- name: Build and export
40+
uses: docker/build-push-action@v5
41+
with:
42+
file: Dockerfile.oss
43+
context: .
44+
tags: nginx-s3-gateway , nginx-s3-gateway:oss
45+
outputs: type=docker,dest=${{ runner.temp }}/oss.tar
46+
- name: Upload artifact
47+
uses: actions/upload-artifact@v3
48+
with:
49+
name: oss
50+
path: ${{ runner.temp }}/oss.tar
51+
retention-days: 1
52+
if-no-files-found: error
2053

54+
test-oss:
55+
runs-on: ubuntu-22.04
56+
needs: build-oss-for-test
57+
steps:
58+
- uses: actions/checkout@v4
2159
- name: Install dependencies
2260
run: sudo apt-get update -qq && sudo apt-get install -y curl wait-for-it
2361
- name: Restore cached binaries
@@ -34,35 +72,60 @@ jobs:
3472
curl --insecure --retry 6 --fail --silent --location "https://dl.min.io/client/mc/release/linux-$(dpkg --print-architecture)/archive/mc.RELEASE.2023-06-19T19-31-19Z.sha256sum" | sha256sum --check -
3573
mv mc.RELEASE.2023-06-19T19-31-19Z mc
3674
chmod +x mc
37-
- name: Save cached binaries
38-
id: cache-binaries-save
39-
uses: actions/cache/save@v3
75+
76+
- name: Download artifact
77+
uses: actions/download-artifact@v3
4078
with:
41-
path: .bin
42-
key: ${{ steps.cache-binaries-restore.outputs.cache-primary-key }}
43-
- name: Run tests - latest njs version
44-
run: ./test.sh --latest-njs --type oss
79+
name: oss
80+
path: ${{ runner.temp }}
81+
- name: Load image
82+
run: |
83+
docker load --input ${{ runner.temp }}/oss.tar
4584
- name: Run tests - stable njs version
4685
run: ./test.sh --type oss
47-
- name: Run tests - stable njs version - unprivileged process
48-
run: ./test.sh --unprivileged --type oss
49-
- name: Run tests - latest njs version - unprivileged process
50-
run: ./test.sh --latest-njs --unprivileged --type oss
51-
52-
build_and_deploy:
53-
runs-on: ubuntu-latest
5486

55-
if: github.ref == 'refs/heads/master'
87+
build-latest-njs-for-test:
88+
runs-on: ubuntu-22.04
89+
needs: test-oss
5690
steps:
57-
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
58-
- uses: actions/checkout@v2
59-
60-
- name: Get current date
61-
id: date
62-
run: echo "date=$(date +'%Y%m%d')" >> $GITHUB_OUTPUT
63-
- name: Configure Github Package Registry
64-
run: echo ${{ secrets.GITHUB_TOKEN }} | docker login docker.pkg.github.com -u $GITHUB_ACTOR --password-stdin
91+
- uses: actions/checkout@v4
92+
- name: Set up Docker Buildx
93+
uses: docker/setup-buildx-action@v3
94+
with:
95+
driver: docker
96+
- name: Download artifact
97+
uses: actions/download-artifact@v3
98+
with:
99+
name: oss
100+
path: ${{ runner.temp }}
101+
- name: Load image
102+
run: |
103+
docker load --input ${{ runner.temp }}/oss.tar
104+
- name: Build and load oss image
105+
uses: docker/build-push-action@v5
106+
with:
107+
file: Dockerfile.latest-njs
108+
context: .
109+
tags: nginx-s3-gateway:latest-njs-oss
110+
load: true
111+
# Save manually here since we need to use the `docker` buildx `driver` but that can't output
112+
# a file that upload-artifact likes.
113+
- name: save image
114+
run: |
115+
docker save nginx-s3-gateway:latest-njs-oss > ${{ runner.temp }}/latest-njs.tar
116+
- name: Upload artifact - latest-njs
117+
uses: actions/upload-artifact@v3
118+
with:
119+
name: latest-njs
120+
path: ${{ runner.temp }}/latest-njs.tar
121+
retention-days: 1
122+
if-no-files-found: error
65123

124+
test-latest-njs:
125+
runs-on: ubuntu-22.04
126+
needs: build-latest-njs-for-test
127+
steps:
128+
- uses: actions/checkout@v4
66129
- name: Install dependencies
67130
run: sudo apt-get update -qq && sudo apt-get install -y curl wait-for-it
68131
- name: Restore cached binaries
@@ -75,79 +138,161 @@ jobs:
75138
run: |
76139
mkdir .bin || exit 0
77140
cd .bin
78-
curl --insecure --retry 6 --fail --silent --location --output mc.RELEASE.2023-06-19T19-31-19Z "https://dl.min.io/client/mc/release/linux-$(dpkg --print-architecture)/archive/mc.RELEASE.2023-06-19T19-31-19Z"
141+
curl --insecure --retry 6 --fail --location --output mc.RELEASE.2023-06-19T19-31-19Z "https://dl.min.io/client/mc/release/linux-$(dpkg --print-architecture)/archive/mc.RELEASE.2023-06-19T19-31-19Z"
79142
curl --insecure --retry 6 --fail --silent --location "https://dl.min.io/client/mc/release/linux-$(dpkg --print-architecture)/archive/mc.RELEASE.2023-06-19T19-31-19Z.sha256sum" | sha256sum --check -
80143
mv mc.RELEASE.2023-06-19T19-31-19Z mc
81144
chmod +x mc
82-
83-
# Run tests and builds image
145+
- name: Download artifact
146+
uses: actions/download-artifact@v3
147+
with:
148+
name: latest-njs
149+
path: ${{ runner.temp }}
150+
- name: Load image
151+
run: |
152+
docker load --input ${{ runner.temp }}/latest-njs.tar
153+
docker tag nginx-s3-gateway:latest-njs-oss nginx-s3-gateway
84154
- name: Run tests - latest njs version
85155
run: ./test.sh --latest-njs --type oss
86-
# latest-njs-oss image push [Github]
87-
- name: Tag container image for Push to github [latest-njs-oss date]
88-
run: docker tag nginx-s3-gateway:latest-njs-oss docker.pkg.github.com/$GITHUB_REPOSITORY/nginx-oss-s3-gateway:latest-njs-oss-${{ steps.date.outputs.date }}
89-
- name: Tag container image for Push to github [latest-njs-oss]
90-
run: docker tag nginx-s3-gateway:latest-njs-oss docker.pkg.github.com/$GITHUB_REPOSITORY/nginx-oss-s3-gateway:latest-njs-oss
91-
- name: Push container image to github [latest-njs-oss date]
92-
run: docker push docker.pkg.github.com/$GITHUB_REPOSITORY/nginx-oss-s3-gateway:latest-njs-oss-${{ steps.date.outputs.date }}
93-
- name: Push container image to github [latest-njs-oss]
94-
run: docker push docker.pkg.github.com/$GITHUB_REPOSITORY/nginx-oss-s3-gateway:latest-njs-oss
95-
96-
# Run tests and builds image
97-
- name: Run tests - stable njs version - unprivileged process
156+
157+
build-unprivileged-for-test:
158+
runs-on: ubuntu-22.04
159+
needs: test-oss
160+
steps:
161+
- uses: actions/checkout@v4
162+
- name: Set up Docker Buildx
163+
uses: docker/setup-buildx-action@v3
164+
with:
165+
driver: docker
166+
- name: Download artifact
167+
uses: actions/download-artifact@v3
168+
with:
169+
name: oss
170+
path: ${{ runner.temp }}
171+
- name: Load image
172+
run: |
173+
docker load --input ${{ runner.temp }}/oss.tar
174+
- name: Build and load oss image
175+
uses: docker/build-push-action@v5
176+
with:
177+
file: Dockerfile.unprivileged
178+
context: .
179+
tags: nginx-s3-gateway:unprivileged-oss
180+
load: true
181+
# Save manually here since we need to use the `docker` buildx `driver` but that can't output
182+
# a file that upload-artifact likes.
183+
- name: save image
184+
run: |
185+
docker save nginx-s3-gateway:unprivileged-oss > ${{ runner.temp }}/unprivileged.tar
186+
- name: Upload artifact - unprivileged
187+
uses: actions/upload-artifact@v3
188+
with:
189+
name: unprivileged
190+
path: ${{ runner.temp }}/unprivileged.tar
191+
retention-days: 1
192+
if-no-files-found: error
193+
194+
test-unprivileged:
195+
runs-on: ubuntu-22.04
196+
needs: build-unprivileged-for-test
197+
steps:
198+
- uses: actions/checkout@v4
199+
- name: Install dependencies
200+
run: sudo apt-get update -qq && sudo apt-get install -y curl wait-for-it
201+
- name: Restore cached binaries
202+
id: cache-binaries-restore
203+
uses: actions/cache/restore@v3
204+
with:
205+
path: .bin
206+
key: ${{ runner.os }}-binaries
207+
- name: Install MinIO Client
208+
run: |
209+
mkdir .bin || exit 0
210+
cd .bin
211+
curl --insecure --retry 6 --fail --location --output mc.RELEASE.2023-06-19T19-31-19Z "https://dl.min.io/client/mc/release/linux-$(dpkg --print-architecture)/archive/mc.RELEASE.2023-06-19T19-31-19Z"
212+
curl --insecure --retry 6 --fail --silent --location "https://dl.min.io/client/mc/release/linux-$(dpkg --print-architecture)/archive/mc.RELEASE.2023-06-19T19-31-19Z.sha256sum" | sha256sum --check -
213+
mv mc.RELEASE.2023-06-19T19-31-19Z mc
214+
chmod +x mc
215+
- name: Download artifact
216+
uses: actions/download-artifact@v3
217+
with:
218+
name: unprivileged
219+
path: ${{ runner.temp }}
220+
- name: Load image
221+
run: |
222+
docker load --input ${{ runner.temp }}/unprivileged.tar
223+
docker tag nginx-s3-gateway:unprivileged-oss nginx-s3-gateway
224+
- name: Run tests - unprivileged
98225
run: ./test.sh --unprivileged --type oss
99-
# unprivileged-oss image push [Github]
100-
- name: Tag container image for Push to github [unprivileged-oss date]
101-
run: docker tag nginx-s3-gateway:unprivileged-oss docker.pkg.github.com/$GITHUB_REPOSITORY/nginx-oss-s3-gateway:unprivileged-oss-${{ steps.date.outputs.date }}
102-
- name: Tag container image for Push to github [unprivileged-oss]
103-
run: docker tag nginx-s3-gateway:unprivileged-oss docker.pkg.github.com/$GITHUB_REPOSITORY/nginx-oss-s3-gateway:unprivileged-oss
104-
- name: Push container image to github [unprivileged-oss date]
105-
run: docker push docker.pkg.github.com/$GITHUB_REPOSITORY/nginx-oss-s3-gateway:unprivileged-oss-${{ steps.date.outputs.date }}
106-
- name: Push container image to github [unprivileged-oss]
107-
run: docker push docker.pkg.github.com/$GITHUB_REPOSITORY/nginx-oss-s3-gateway:unprivileged-oss
108-
109-
# Run tests and builds image
110-
- name: Run tests - stable njs version
111-
run: ./test.sh --type oss
112-
# oss image push [Github]
113-
- name: Tag container image for Push to github [oss date]
114-
run: docker tag nginx-s3-gateway:oss docker.pkg.github.com/$GITHUB_REPOSITORY/nginx-oss-s3-gateway:latest-${{ steps.date.outputs.date }}
115-
- name: Tag container image for Push to github [oss]
116-
run: docker tag nginx-s3-gateway:oss docker.pkg.github.com/$GITHUB_REPOSITORY/nginx-oss-s3-gateway:latest
117-
- name: Push container image to github [oss date]
118-
run: docker push docker.pkg.github.com/$GITHUB_REPOSITORY/nginx-oss-s3-gateway:latest-${{ steps.date.outputs.date }}
119-
- name: Push container image to github [oss latest]
120-
run: docker push docker.pkg.github.com/$GITHUB_REPOSITORY/nginx-oss-s3-gateway:latest
121-
# Login to Docker Hub
226+
227+
# After the tests are done, build multiarch and push to both github packages and dockerhub if we are on master/main
228+
tag-and-push:
229+
runs-on: ubuntu-22.04
230+
needs: [test-oss, test-latest-njs, test-unprivileged]
231+
232+
if: |
233+
github.ref == 'refs/heads/master' ||
234+
github.ref == 'refs/heads/main'
235+
steps:
236+
- uses: actions/checkout@v4
237+
- name: Get current date
238+
id: date
239+
run: echo "date=$(date +'%Y%m%d')" >> $GITHUB_OUTPUT
240+
- name: Set up QEMU
241+
uses: docker/setup-qemu-action@v3
242+
- name: Set up Docker Buildx
243+
uses: docker/setup-buildx-action@v3
244+
with:
245+
platforms: linux/amd64,linux/arm64
122246
- name: Login to Docker Hub
123-
uses: docker/login-action@v1
247+
uses: docker/login-action@v3
124248
with:
125249
username: ${{ secrets.DOCKER_HUB_USERNAME }}
126250
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
127-
# latest-njs-oss image push [Docker Hub]
128-
- name: Tag container image for Push to Docker Hub [latest-njs-oss date]
129-
run: docker tag nginx-s3-gateway:latest-njs-oss nginxinc/nginx-s3-gateway:latest-njs-oss-${{ steps.date.outputs.date }}
130-
- name: Tag container image for Push to Docker Hub [latest-njs-oss]
131-
run: docker tag nginx-s3-gateway:latest-njs-oss nginxinc/nginx-s3-gateway:latest-njs-oss
132-
- name: Push container image to Docker Hub [latest-njs-oss date]
133-
run: docker push nginxinc/nginx-s3-gateway:latest-njs-oss-${{ steps.date.outputs.date }}
134-
- name: Push container image to Docker Hub [latest-njs-oss]
135-
run: docker push nginxinc/nginx-s3-gateway:latest-njs-oss
136-
# unprivileged-oss image push [Docker Hub]
137-
- name: Tag container image for Push to Docker Hub [unprivileged-oss date]
138-
run: docker tag nginx-s3-gateway:unprivileged-oss nginxinc/nginx-s3-gateway:unprivileged-oss-${{ steps.date.outputs.date }}
139-
- name: Tag container image for Push to Docker Hub [unprivileged-oss]
140-
run: docker tag nginx-s3-gateway:unprivileged-oss nginxinc/nginx-s3-gateway:unprivileged-oss
141-
- name: Push container image to Docker Hub [unprivileged-oss date]
142-
run: docker push nginxinc/nginx-s3-gateway:unprivileged-oss-${{ steps.date.outputs.date }}
143-
- name: Push container image to Docker Hub [unprivileged-oss]
144-
run: docker push nginxinc/nginx-s3-gateway:unprivileged-oss
145-
# oss image push [Docker Hub]
146-
- name: Tag container image for Push to Docker Hub [oss date]
147-
run: docker tag nginx-s3-gateway:oss nginxinc/nginx-s3-gateway:latest-${{ steps.date.outputs.date }}
148-
- name: Tag container image for Push to Docker Hub [oss]
149-
run: docker tag nginx-s3-gateway:oss nginxinc/nginx-s3-gateway:latest
150-
- name: Push container image to Docker Hub [oss date]
151-
run: docker push nginxinc/nginx-s3-gateway:latest-${{ steps.date.outputs.date }}
152-
- name: Push container image to Docker Hub [oss latest]
153-
run: docker push nginxinc/nginx-s3-gateway:latest
251+
- name: Login to GitHub Container Registry
252+
uses: docker/login-action@v3
253+
with:
254+
registry: ghcr.io
255+
username: ${{ github.repository_owner }}
256+
password: ${{ secrets.GITHUB_TOKEN }}
257+
258+
- name: Build and push image [oss]
259+
uses: docker/build-push-action@v5
260+
with:
261+
file: Dockerfile.oss
262+
context: .
263+
push: true
264+
platforms: linux/amd64,linux/arm64
265+
provenance: false
266+
tags: |
267+
ghcr.io/${{ github.repository }}/nginx-oss-s3-gateway:latest-${{ steps.date.outputs.date }}
268+
ghcr.io/${{ github.repository }}/nginx-oss-s3-gateway:latest
269+
nginxinc/nginx-s3-gateway:latest-${{ steps.date.outputs.date }}
270+
nginxinc/nginx-s3-gateway:latest
271+
272+
- name: Build and push image [latest-njs]
273+
uses: docker/build-push-action@v5
274+
with:
275+
file: Dockerfile.latest-njs
276+
context: .
277+
push: true
278+
platforms: linux/amd64,linux/arm64
279+
provenance: false
280+
tags: |
281+
ghcr.io/${{ github.repository }}/nginx-oss-s3-gateway:latest-njs-oss-${{ steps.date.outputs.date }}
282+
ghcr.io/${{ github.repository }}/nginx-oss-s3-gateway:latest-njs-oss
283+
nginxinc/nginx-s3-gateway:latest-njs-oss-${{ steps.date.outputs.date }}
284+
nginxinc/nginx-s3-gateway:latest-njs-oss
285+
286+
- name: Build and push image [unprivileged]
287+
uses: docker/build-push-action@v5
288+
with:
289+
file: Dockerfile.unprivileged
290+
context: .
291+
push: true
292+
platforms: linux/amd64,linux/arm64
293+
provenance: false
294+
tags: |
295+
ghcr.io/${{ github.repository }}/nginx-oss-s3-gateway:unprivileged-oss-${{ steps.date.outputs.date }}
296+
ghcr.io/${{ github.repository }}/nginx-oss-s3-gateway:unprivileged-oss
297+
nginxinc/nginx-s3-gateway:unprivileged-oss-${{ steps.date.outputs.date }}
298+
nginxinc/nginx-s3-gateway:unprivileged-oss

0 commit comments

Comments
 (0)