7

I am using build-push-action in Github Actions (with buildx) to build Docker image and push it to AWS ECR registry. Before I push the image (unfortunately, it has several GB), I want to use the image to run containerized Python tests.

build-push-action has docs on this exact goal - their example suggests to build the image, load it to Docker, use docker run ... and then push. However, while it takes 7 minutes to build the image (from downloaded cache), it takes another 7 minutes to load it to Docker - so it is doubling the time of the CI pipeline. The example from docs:


      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v1
      - name: Build and export to Docker
        uses: docker/build-push-action@v2
        with:
          context: .
          load: true
          tags: ${{ env.TEST_TAG }}
      - name: Test
        run: |
          docker run --rm ${{ env.TEST_TAG }}
      - name: Build and push
        uses: docker/build-push-action@v2
        with:
          context: .
          platforms: linux/amd64,linux/arm64
          push: true
          tags: user/app:latest

Are you aware of any way to speed up/shorten the loading time or how to run docker run on the image in other way? It seems that the problem is, that for the loading, the image is first exported to tarball and then loaded to docker, which is not accelerated by any sort of cache. (Pushing the image to ECR and pulling it back to docker is almost faster.) However, without the loading, I have not been able to get the image.

Run tests inside Dockerfile is a quick workaround that works. But then we get to the same issue - how to get your hands on the output files (e.g. coverage stats) without dirty hacks.

Thanks.

For completeness, adding my real-world example where I want to place the tests:


- name: Login to Amazon ECR
  id: login-ecr
  uses: aws-actions/amazon-ecr-login@v1

- name: Set up Docker Buildx
  uses: docker/setup-buildx-action@v1

- name: Build Image and Push to ECR
  uses: docker/build-push-action@v2
  env:
    ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
  with:
    context: .
    push: true
    build-args: |
      build_id=${{ github.sha }}
    tags: |
      ${{ env.ECR_REGISTRY }}/${{ env.ECR_REPOSITORY }}:latest
    cache-from: type=registry,ref=${{ env.ECR_REGISTRY }}/${{ env.ECR_REPOSITORY }}:latest
    cache-to: type=inline
Marek
  • 815
  • 8
  • 19

1 Answers1

1

My question was partially based on wrong assumptions about BuildX and its role in the example workflow. (I thought you must use BuildX to use BuildKit in the action and our Dockerfile used a few BuildKit features)

I found answer to my question partially in Docker BuildX image not showing in docker image ls and a blog post Multi-arch build and images, the simple way.

By using setup-buildx-action step, BuildKit builds multi-platform images that does not get directly loaded into docker (since you need a single-platform image to be loaded). We currently don't need multi-platform builds and we prefer the speed of our CI. So removing setup-buildx-action from the workflow caused, that the single-platform image appears in docker images and it can be immediately used for running the tests (with no obvious time overhead for loading single-platform images to the docker).

I.e. commenting out this step helped:

- name: Set up Docker Buildx
  uses: docker/setup-buildx-action@v1

Once our pipeline gets more complicated and lengthy, we may start building multi-platform images again, but for now, saving 6 minutes on the run makes more sense.

Note: Feel free to add your answers/comments. I believe there is a better solution than removing buildx setup and also I wonder if there is a smart solution for building multi-platform solutions and getting the image loaded to docker images fast.

Marek
  • 815
  • 8
  • 19