Docker push to AWS ECR hangs immediately and times out

Question

I'm trying to push my first docker image to ECR. I've followed the steps provided by AWS and things seem to be going smoothly until the final push which immediately times out. Specifically, I pass my aws ecr credentials to docker and get a "login succeeded" message. I then tag the image which also works. pushing to the ecr repo I get no error message, just the following:

The push refers to repository [xxxxxxxxxxx.dkr.ecr.ca-central-1.amazonaws.com/reponame]
714c1b96dd83: Retrying in 1 second 
d2cdc77dd068: Retrying in 1 second 
30aad807caf5: Retrying in 1 second 
0559774c4ea2: Retrying in 1 second 
285b8616682f: Retrying in 1 second 
4aeea0ec2b15: Waiting 
1b1312f842d8: Waiting 
c310009e0ef3: Waiting 
a48777e566d3: Waiting 
2a0c9f28029a: Waiting 
EOF

It tries a bunch of times and then exits with no message. Any idea what's wrong?

Answer 1

I figured out my issue. I wasn't using the correct credentials. I had a personal AWS account as my default credentials and needed to add my work profile to my credentials.

EDIT
If you have multiple aws profiles, you can mention the profile name at the docker login as below (assuming you have done aws configure --profile someprofile at earlier day),

aws ecr get-login-password --region us-east-1 --profile someprofile | docker login ....

Answer 2

You will get the same behaviour if you forget to create ECR repo before pushing.

Use CloudTrail to get a clue what is wrong.

Answer 3

Make sure the name of your repository is the same name as your images. image:latest 756839881602.dkr.ecr.us-east-1.amazonaws.com/image:latest in this case my repository name is image and my image name is image as well. This worked for me.

Answer 4

Also make sure that you have configured correct policy for your user — for example, AmazonEC2ContainerRegistryFullAccess.

Answer 5

In my case, the repository I wanted to push to didn't exist (For example, I tried pushing to my-app/backend:latest but only the my-app/cms repository exists). So make sure your repository exists in the AWS ECR Console in the right region. The error returned from AWS CLI (EOF) didn't help at all.

Answer 6

Check your aws permissions. In addition to AmazonEC2ContainerRegistryFullAccess permission, below actions has to be granted for the correct resource. Especially check "arn:aws:ecr:${REGION}:${ACCOUNT_ID}:repository/{$REGISTRY_NAME}" part.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ecr:BatchGetImage",
        "ecr:BatchCheckLayerAvailability",
        "ecr:CompleteLayerUpload",
        "ecr:DescribeImages",
        "ecr:DescribeRepositories",
        "ecr:GetDownloadUrlForLayer",
        "ecr:InitiateLayerUpload",
        "ecr:ListImages",
        "ecr:PutImage",
        "ecr:UploadLayerPart"
      ],
      "Resource": "arn:aws:ecr:${REGION}:${ACCOUNT_ID}:repository/{$REGISTRY_NAME}"
    },
    {
      "Effect": "Allow",
      "Action": "ecr:GetAuthorizationToken",
      "Resource": "*"
    }
  ]
}

Answer 7

Assuming you authenticated successfully to AWS and you have permissions to read, write to ECR, check if the repository does exist

aws ecr describe-repositories --repository-name reponame

If you catch an error RepositoryNotFoundException, then you will create to that repository with the following command

aws ecr create-repository --repository-name reponame

After that, try to push again, it will be fine!

Answer 8

I have to add for anyone else encountering this problem. Go to IAM and make sure you have put permissions. I don't want to say how long I wasted before figuring that out.

Edit to help @zac's answer: The policies that need to be attached are AmazonEC2ContainerRegistryFullAccess and AWSAppRunnerServicePolicyForECRAccess

Answer 9

In my case it was related to MFA (Multi-Factor-Authentication). I had to create a session token. The docker login seemed to be successful, but pushing does not work.

The following script is doing all for you and creates a aws profile "mfa" used to login: get_mfa_credentials.py

After executing, you can login with:

aws ecr get-login-password --region <YOUR_REGION> --profile mfa | docker login --username AWS --password-stdin <Your_REPO>

I do not know who wrote it, but I'm very grateful to this guy.

And thanks to AWS for bad tools that do not help.

Answer 10

For those who tried the solution above, and it didn't work, make sure the image name your are pushing is the same as the repository name.

Answer 11

Ensure you are using the correct profile and that the repository exists

Command to login with profile: aws ecr get-login-password --region <region> --profile=<profile-name> | docker login --username AWS --password-stdin <aws-account-id>.dkr.ecr.eu-west-1.amazonaws.com

Command to create repo if it does not exists: aws ecr describe-repositories --repository-names ${REPO_NAME} || aws ecr create-repository --repository-name ${REPO_NAME}(source)

Answer 12

I also was able to login to the registry, yet the pushing of the image would just timeout.

The solution for me was to add AmazonEC2ContainerRegistryFullAccess to my IAM user.

After adding that permission to my IAM user account, I could docker push to the ECS registry just fine.

Answer 13

The issue was resolved, when I created the docker repository first (in ECR) and then pushed it to ECR. Remember to create the docker repository, before running the docker push command.

Answer 14

For me, I misconfigured the tag name like this:

$ docker tag <image_name>:latest xxxxxxxxxxx.dkr.ecr.ca-central-1.amazonaws.com/reponame/<additional_string>:latest
$ docker push xxxxxxxxxxx.dkr.ecr.ca-central-1.amazonaws.com/reponame/<additional_string>:latest
...timeout...

And I found the <additional_string> should be deleted:

$ docker tag <image_name>:latest xxxxxx.amazonaws.com/reponame:latest
$ docker push xxxxxx.amazonaws.com/reponame:latest
...success!...

Answer 15

If anyone is still stuck with the issue. I would highly recommend watching this short vid https://www.youtube.com/watch?v=89ZeXaZEf80&ab_channel=IdenticalCloud

Here are the steps I took to fix the issue (if you prefer not to watch the video):

1. Create a new IAM user with "Access keys" checked
2. Under permissions, click on "attach existing policies directly" and choose "AmazonEC2ContainerRegistryFullAccess"
3. Download the CSV file
4. Run "AWS configure" on your terminal and pass in the credentials from the CSV file
5. Set the location to the location you created your ECR (mine was us-east-1)
6. Go to ECR and follow the steps to push the image

Answer 16

Please check cloud trail event logs , this is where all the api issues are clearly highlighted .

In my case it was because i had a - in my image name and hence it was throwing the following error in the cloud trail logs

"The repository with name 'myimage-nginx' does not exist in the registry with id '516583196897'

Please note the - in the image name.

Fixing the image name to remove the - resolved the issue for me.

Commands

docker tag nginx:latest 516583196897.dkr.ecr.ap-south-1.amazonaws.com/myimage:latest

docker push 516583196897.dkr.ecr.ap-south-1.amazonaws.com/myimage:latest

Answer 17

For me, I had to delete the stack and re-deploy the stack. Then, I was able to push the docker image to ECR.

Answer 18

In my case I was creating the repo in us-east-2 and attempting to push to us-east-1, so docker couldn't find it.

Answer 19

Make sure your assumed aws role has the ability to push images to AWS ECR. Easiest is to check the role via the command:

aws sts get-caller-identity --profile=saml

Answer 20

I was following this documentation and hit this error. What addressed the problem was using the repository id instead of the account name.

aws ecs create-repository creates a repo, returning a repositoryUri. Then, the docker login, docker tag and docker push should be done using that repository url instead of the user one.

Answer 21

I had this problem with sam deploy

• sam delete --stack-name ...
• sam deploy --guided

worked for me

Answer 22

Haven't found any solution?

Maybe you are falling in the use case I had.

identifiers:

1. You working on different AWS accounts on the same terminal.

2. When jumping between AWS accounts, you're using export AWS_PROFILE (directly or via some tool or command) in the terminal level to export the profile.

3. You are able to push sometimes, and sometimes not.

4. You do manage to login with aws ecr get-login-password.

What was the cause?

When I switched between AWS accounts, although I had a valid session token for each account, BUT I forgot sometimes to export AWS_PROFILE back the relevant profile.

Answer 23

In my case, it was a configuration issue, aws cli was configured for my other aws account and i was trying to push my image in my mother account

Figured it out with aws configure, and changed the necessary credentials.

Answer 24

In my case I didn't specify a region in my policy:

- Effect: Allow
  Action:
    - ecr:*
  Resource: !Sub "arn:aws:ecr::${AWS::AccountId}:repository/my-image"

Should be:

- Effect: Allow
  Action:
    - ecr:*
  Resource: !Sub "arn:aws:ecr:eu-central-1:${AWS::AccountId}:repository/my-image"

Answer 25

The easiest way I have found is. Go to the console and create a repository. Click on the created repo to see the view push commands button. click and it will list all commands you need to build and push the image into the repo. you don't need to do any changes.