1

We are deploying an application with about 200MB of files to Elastic Beanstalk. We have everything in git and have used eb init (v. 3.0.9 with Python 2.7.6) and friends to set up deploys.

When we use eb deploy everything deploys properly but eb transfers all 200MB of files every time. We were assuming that since everything is set up with git, it should only transfer changes.

We have the same problem when using aws.push with eb version 2.

What is the best way to quickly deploy a large application like this? Depending on the Internet connection, the transfer takes 5-10 minutes. We'd like it to take less than one minute.

Edit:

We discovered that eb v3 transfers all files by design--see this AWS Developer Forum thread. I'd be interested to know if anyone has a better way to do deploys. Maybe something that takes advantage of git changesets.

kendsnyder
  • 764
  • 7
  • 12
  • which files takes so much space in your git? why not store them in s3 instead? – Tal Nov 13 '14 at 08:37
  • @Tal 3rd party code libraries take up 95% of the space. I don't know a way to keep them elsewhere but copy them into the application folder after deploy while retaining the ability to update those libraries periodically. If you have any suggestions on that, I'm all ears. – kendsnyder Nov 13 '14 at 20:33
  • What language are you using? Typically you can have the instances pull dependencies rather having to include them in your application bundle that you send. – Nick Humrich Nov 14 '14 at 18:54
  • The language is PHP. It is possible to put post-deploy scripts in .ebextensions/scripts/post-deploy but AFAIK that usage is not officially supported. There is no guarantee that those scripts will run at all, and if they do run that they will finish before the server is put back into the cluster. We don't want the application to break because the dependencies are not downloaded yet. We also found that pre-deploy scripts that modify the source tree don't work because the uploaded application files are copied to /var/app/current after the pre-deploy scripts run. If that is wrong, I'm all ears. – kendsnyder Nov 15 '14 at 20:06
  • actually pulling the libraries by using a pre-deploy script is exactly the way to do it. it is also guaranteed by EB to work. – Tal Nov 16 '14 at 08:24
  • @Tal, do you know if any code examples with that approach? – kendsnyder Nov 17 '14 at 16:35
  • kendsnyder - Any updates to your findings. I'm looking to deploy a large application and wondering if you were able to solve the issue. – alloyking Jan 07 '15 at 14:30
  • @alloyking Not much new to report. As that AWS forum thread mentions, `eb` uses `git archive`. We were able to reduce the size of the archive by creating a .gitattributes file that lists files to exclude from the archive (e.g. readme, unit tests, etc.). See here: http://alexbilbie.com/2012/11/exclude-objects-with-gitattributes/ . Ideally I'd like to see AWS better document deploy-related hooks. We've found trial and error the best way to see the order of the hooks and when files get copied over. – kendsnyder Jan 08 '15 at 17:55
  • A very similar question has been discussed here: [Elastic Beanstalk “git aws.push” only commited difference?](http://stackoverflow.com/questions/26941775/elastic-beanstalk-git-aws-push-only-commited-difference) – Simon East Sep 08 '15 at 03:22

0 Answers0