Goal: Build a continuous integration pipeline for a spring boot application that runs JUnit tests, packages, builds a docker image, and finally pushes that image to Amazon Elastic Container Registry.
The pipeline that I have built is able to run the maven test phase, run maven package, but then complains while executing the task for building the docker image.
Below is a screenshot of the pipeline.
As you can see the build-and-push job partially fails. Below is the screenshot of tasks contained inside the build-and-push job.
And we see the error version is missing from the previous step.
My pipeline looks like this
resources:
- name: session-management-service-repo
type: git
icon: github
source:
branch: develop
uri: ((source_url))
username: ((myusername))
password: ((mypassowrd)
- name: ecr-docker-reg
type: registry-image
icon: docker
source:
aws_access_key_id: ((access_key_id))
aws_secret_access_key: ((secret_access_key))
aws_region: ((region))
repository: srm-session-management-service
tag: latest
resource_types:
- name: registry-image
type: docker-image
source:
repository: registry:5000/srm/registry-image-resource
insecure_registries:
- registry:5000
jobs:
- name: test
public: true
plan:
- get: session-management-service-repo
trigger: true
- task: mvn-test-task
file: session-management-service-repo/ci/tasks/maven-test.yml
- name: build-and-push
public: true
serial: true
plan:
- get: session-management-service-repo
trigger: true
passed: [test]
- task: mvn-package-task
file: session-management-service-repo/ci/tasks/maven-package.yml
- task: build-image-task
privileged: true # oci-build-task must run in a privileged container
file: session-management-service-repo/ci/tasks/build-image.yml
- put: ecr-docker-reg
params: {image: image/image.tar}
Here I have built a custom resource type that is extended from concourse/registry-image-resource. Basically, I wanted to include some certificates in the resource so that it does not face any problem while uploading the image to ECR as I run behind a proxy. So, the docker file for this custom resource looks like below. I build the image from this dockerfile and push the image running on the same server where the concourse is running, thus in a private docker registry. Later in the pipeline, as you can see I pull this resource type from the custom docker registry...check ecr-docker-reg in the resources section. (This is what I am trying to do.)
FROM concourse/registry-image-resource
ARG HTTP_PROXY=http://username:password@myhost:port
ARG HTTPS_PROXY=http://username:password@myhost:port
ARG NO_PROXY=localhost,*.myhost.com,127.0.0.1,.myhost.com
ENV http_proxy=${HTTP_PROXY}
ENV https_proxy=${HTTPS_PROXY}
ENV no_proxy=${NO_PROXY}
ENV HTTP_PROXY=${HTTP_PROXY}
ENV HTTPS_PROXY=${HTTPS_PROXY}
ENV NO_PROXY=${NO_PROXY}
COPY certificates/Cert-CA-bundle.crt /etc/pki/tls/certs/ca-bundle.crt
#RUN apk update && apk add --no-cache curl
The maven package task and the accompanying script looks like this
---
platform: linux
image_resource:
type: docker-image
source:
repository: maven
inputs:
- name: session-management-service-repo
run:
path: /bin/sh
args: ["./session-management-service-repo/ci/scripts/maven-package.sh"]
outputs:
- name: session-management-service-repo-out
maven package script
#!/bin/bash
set -e
mvn -version
cd session-management-service-repo
cp -f ci/assets/maven/settings.xml /usr/share/maven/conf/settings.xml
mvn clean package -DskipTests=true
cp -a * ../session-management-service-repo-out
And the build-image-task looks like this
---
platform: linux
image_resource:
type: registry-image
source:
repository: concourse/oci-build-task
inputs:
- name: session-management-service-repo-out
outputs:
- name: image
params:
CONTEXT: session-management-service-repo-out
run:
path: build
Note: One thing to note here is that this error that I started to get is when I used my custom resource type. Before using my custom resource type I did not face this "version is missing from previous step" error, rather it was something like below which I only got while pushing the docker image and not while building the image, so I was successfully able to build the image. But as you can see that it the certificate error, I therefore decided to use custom resource type that has the needed certificates included.
selected worker: 1b0fd33bcd2b
WARN[0000] ECR integration is experimental and untested
ERRO[0000] failed to authenticate to ECR: RequestError: send request failed
caused by: Post "https://api.ecr.eu-central-1.amazonaws.com/": x509: certificate signed by unknown authority
ERRO[0000] cannot authenticate with ECR
My pipeline before using custom resource type was almost similar, just that it did not contain the resource_types section
resources:
- name: session-management-service-repo
type: git
icon: github
source:
branch: develop
uri: ((source_url))
username: ((myusername))
password: ((mypassword))
- name: ecr-docker-reg
type: registry-image
icon: docker
source:
aws_access_key_id: ((access_key))
aws_secret_access_key: ((secret_access_key))
aws_region: ((region))
repository: srm-session-management-service
tag: latest
jobs:
- name: test
public: true
plan:
- get: session-management-service-repo
trigger: true
- task: mvn-test-task
file: session-management-service-repo/ci/tasks/maven-test.yml
- name: build-and-push
public: true
serial: true
plan:
- get: session-management-service-repo
trigger: true
passed: [test]
- task: mvn-package-task
file: session-management-service-repo/ci/tasks/maven-package.yml
- task: build-image-task
privileged: true # oci-build-task must run in a privileged container
file: session-management-service-repo/ci/tasks/build-image.yml
- put: ecr-docker-reg
params: {image: image/image.tar}
I am not able to figure out what am I missing or where am I going wrong. Any suggestion would be grateful. Thanks
