switch to rules_img and add image amd64/v3 variant

[malt3]

This PR switches from rules_pkg and rules_oci to rules_img, thereby halving the container image size and offering an image that's optimized for x86-64-v3.

Space savings

Before, rules_pkg would not optimize the runfiles of the go_binary target in any way. Since Bazel binaries usually have a runfiles tree that contains the application itself, the image ends up containing the same data twice:

$ tar -tvf rules_oci_layout/blobs/sha256/8c1e0997b01c5215367a88561471650d31748259cefddd94ce78744c4eb5a363
drwxr-xr-x 0/0               0 2000-01-01 01:00 app/
drwxr-xr-x 0/0               0 2000-01-01 01:00 app/bb_storage.runfiles/
drwxr-xr-x 0/0               0 2000-01-01 01:00 app/bb_storage.runfiles/_main/
drwxr-xr-x 0/0               0 2000-01-01 01:00 app/bb_storage.runfiles/_main/cmd/
drwxr-xr-x 0/0               0 2000-01-01 01:00 app/bb_storage.runfiles/_main/cmd/bb_storage/
drwxr-xr-x 0/0               0 2000-01-01 01:00 app/bb_storage.runfiles/_main/cmd/bb_storage/bb_storage_/
-rwxr-xr-x 0/0        49500344 2000-01-01 01:00 app/bb_storage.runfiles/_main/cmd/bb_storage/bb_storage_/bb_storage
-rwxr-xr-x 0/0        49500344 2000-01-01 01:00 app/bb_storage
-r-xr-xr-x 0/0             117 2000-01-01 01:00 app/bb_storage.repo_mapping
-r-xr-xr-x 0/0             117 2000-01-01 01:00 app/bb_storage.runfiles/_repo_mapping

With rules_img, we can deduplicate multiple copies of the Go binary:

$ ❯ tar -tvf rules_img_layout/blobs/sha256/353f76ceb3351f5e21cfdd704e5fac83dd0d225ba53f9bbc6dbeec2673e6c535
-rwxr-xr-x 0/0        49500344 1970-01-01 01:00 .cas/blob/851ee6a2ad126fc224a8b313495720dcfd1b1d1b94ff907c54e68c7df6b9c739
hrwxr-xr-x 0/0               0 1970-01-01 01:00 app/bb_storage link to .cas/blob/851ee6a2ad126fc224a8b313495720dcfd1b1d1b94ff907c54e68c7df6b9c739
drwxr-xr-x 0/0               0 1970-01-01 01:00 app/bb_storage.runfiles/
hrwxr-xr-x 0/0               0 1970-01-01 01:00 app/bb_storage.runfiles/_main/cmd/bb_storage/bb_storage_/bb_storage link to .cas/blob/851ee6a2ad126fc224a8b313495720dcfd1b1d1b94ff907c54e68c7df6b9c739

Architecture variants

The existing image index offers variants for linux/arm64 and linux/amd64. With this change, we also add a linux/amd64/v3 image variant that can be picked up by a compatible container runtime automatically.
This variant is compiled with the GOAMD64 value set to v3, enabling newer CPU instructions that may accelerate CPU-intensive operations such as hashing.
Older x86 CPUs can continue to use the baseline variant (linux/amd64).

I'm currently unable to provide a wide range of benchmarks (I'm travelling), but users are invited to test the image I pushed to ghcr.io/malt3/bb-storage:amd64v3 and report results.
I did compare the resulting binary with diffoscope and was able to verify that the linux/amd64/v3 binary does indeed contain instructions from newer architecture variants.

[aspect-workflows]

Test

All tests were cache hits

30 tests (100.0%) were fully cached saving 5s.

[EdSchouten]

This means we only end up publishing container images that use this. However, we also publish regular binaries through our CI pipeline. Can you make sure that those also get built for x86_64-v3?

Slightly annoying that the platforms module doesn't offer standardized definitions for this. A bit silly that both rules_go and rules_img need to declare their own constraints.

[malt3]

However, we also publish regular binaries through our CI pipeline. Can you make sure that those also get built for x86_64-v3?

I'll take a look. The workflow in .github/workflows/main.yaml is not exactly trivial, but I'll give it a shot.

[EdSchouten]

Note that the source of truth of those files lives in tools/github_workflows. Just "bazel build" and copy the outputs into .github.

[malt3]

Happy to clean this up using the Jsonnet template as suggested, but before I do:

CI seems to fail with "no space left on device".
Are we sure that we want to build this extra platform? And if so, should we just call "bazel clean" between platforms? I believe the sharing of actions between platforms is very limited, so cleaning Bazel's output base when switching platforms might be a good tradeoff.

[EdSchouten]

The fact that we have similar constraints both on the rules_go and rules_img side makes me believe @platforms should be extended to offer those. Maybe it's worth filing an issue/submitting a PR there as well?

[malt3]

I commented on an existing issue: bazelbuild/platforms#13 (comment)

While I agree that this would be better, I'm unsure if a consensus can be reached over in the platforms repo.