Update Docker pipeline to use the new Docker buildkit engine

[alessfg]

Is your feature request related to a problem? Please describe

The pipeline in this repo uses the "legacy" Docker engine to build and test images. Since Feb 2023 (and Docker engine release 23.0), Docker buildkit has become the default Docker build engine. Docker buildkit introduces multiple build improvements and features, and as such, it would make sense to update the pipeline in this repo to use Docker buildkit.

Describe the solution you'd like

The pipeline in this repo should use Docker buildkit as the build engine.

Describe alternatives you've considered

N/A

Additional context

N/A

[yosifkit]

The workflow uses the docker-library/bashbrew action that we maintain in the Docker Official Images, we do need to do some updates there, but it is only used test builds periodically and on PRs. It is not how the images are built and pushed to Docker Hub.

We have been actively migrating the Docker Official Images builds to a newer process that uses buildkit by default (see https://github.com/docker-library/meta-scripts and https://github.com/docker-library/meta repos). We are being cautious by only moving a few at a time so as to not cause needless churn or problems for users of the images. When it is complete, we plan to update the action to more closely follow how they are built for Docker Hub. You can see some of the additions of the new build system by inspecting the index and manifest of a migrated image to see annotations and attached sboms and provenance data (subject to change):

• https://explore.ggcr.dev/?image=nginx%3Alatest (image index)
• https://explore.ggcr.dev/?image=nginx@sha256:db5e49f40979ce521f05f0bc9f513d0abacce47904e229f3a95c2e6d9b47f244 (amd64 image manifest)

There are few features of buildkit that are an improvement of classic builder. Even after years of it being developed as a replacement of the classic builder, it still has bugs and incompatible behavior changes (like these recently: moby/buildkit#4974, moby/buildkit#5072, moby/buildkit#5066). So, this is why we have been cautious to adopt it for all of the Official Images builds.

[thresheek]

Thanks for the great insight @yosifkit. I don't think we have a deal-breaking feature we require from buildkit for now - just some cosmetic stuff as in #906 which would be nice to have.

I think we can wait until bashbrew action supports it.