42

I'm having an issue deploying our rails app.. I created a hook like the example on the AWS blog howto http://ruby.awsblog.com/post/Tx2AK2MFX0QHRIO/Deploying-Ruby-Applications-to-AWS-Elastic-Beanstalk-with-Git like:

packages:
  yum:
    git: []

even I run a bundle package to create vendor/cache to have all the gems there... and still getting: git://github.com/refinery/refinerycms-search.git (at 2-0-stable) is not checked out. Please run bundle install (Bundler::GitError)

any help will be nice, we trying to move all our apps to EB. but seens that git does not install or something is going on.. I need git on the EB ec2 instance it creates.

StackTrace:

Error message:
git://github.com/refinery/refinerycms-search.git (at 2-0-stable) is not checked out.         Please run `bundle install` (Bundler::GitError)
Exception class:
PhusionPassenger::UnknownError
Application root:
/var/app/current
Backtrace:
#   File    Line    Location
0   /usr/share/ruby/1.9/gems/1.9.1/gems/bundler-1.2.1/lib/bundler/source.rb 801 in        `rescue in load_spec_files'
1   /usr/share/ruby/1.9/gems/1.9.1/gems/bundler-1.2.1/lib/bundler/source.rb 799 in   `load_spec_files'
2   /usr/share/ruby/1.9/gems/1.9.1/gems/bundler-1.2.1/lib/bundler/source.rb 381 in `local_specs'
3   /usr/share/ruby/1.9/gems/1.9.1/gems/bundler-1.2.1/lib/bundler/source.rb 774 in `specs'
4   /usr/share/ruby/1.9/gems/1.9.1/gems/bundler-1.2.1/lib/bundler/definition.rb 174 in `block in resolve'
5   /usr/share/ruby/1.9/gems/1.9.1/gems/bundler-1.2.1/lib/bundler/definition.rb 172 in `each'
6   /usr/share/ruby/1.9/gems/1.9.1/gems/bundler-1.2.1/lib/bundler/definition.rb 172 in `resolve' 
7   /usr/share/ruby/1.9/gems/1.9.1/gems/bundler-1.2.1/lib/bundler/definition.rb 113 in `specs'
8   /usr/share/ruby/1.9/gems/1.9.1/gems/bundler-1.2.1/lib/bundler/definition.rb 158 in `specs_for'
9   /usr/share/ruby/1.9/gems/1.9.1/gems/bundler-1.2.1/lib/bundler/definition.rb 147 in `requested_specs'
10  /usr/share/ruby/1.9/gems/1.9.1/gems/bundler-1.2.1/lib/bundler/environment.rb    23  in `requested_specs'
11  /usr/share/ruby/1.9/gems/1.9.1/gems/bundler-1.2.1/lib/bundler/runtime.rb    11   in `setup'
12  /usr/share/ruby/1.9/gems/1.9.1/gems/bundler-1.2.1/lib/bundler.rb    116 in `setup'
13  /var/lib/passenger-standalone/3.0.17-x86_64-ruby1.9.3-linux-gcc4.4.6-1002/support/lib/phusion_passenger/utils.rb    326 in `prepare_app_process'
14  /var/lib/passenger-standalone/3.0.17-x86_64-ruby1.9.3-linux-gcc4.4.6-1002/support/lib/phusion_passenger/rack/application_spawner.rb 156 in `block in initialize_server'
15  /var/lib/passenger-standalone/3.0.17-x86_64-ruby1.9.3-linux-gcc4.4.6-1002/support/lib/phusion_passenger/utils.rb    563 in `report_app_init_status'
16  /var/lib/passenger-standalone/3.0.17-x86_64-ruby1.9.3-linux-gcc4.4.6-1002/support/lib/phusion_passenger/rack/application_spawner.rb 154 in `initialize_server'
17  /var/lib/passenger-standalone/3.0.17-x86_64-ruby1.9.3-linux-gcc4.4.6-  1002/support/lib/phusion_passenger/abstract_server.rb    204 in `start_synchronously'
18  /var/lib/passenger-standalone/3.0.17-x86_64-ruby1.9.3-linux-gcc4.4.6-1002/support/lib/phusion_passenger/abstract_server.rb  180 in `start'
19  /var/lib/passenger-standalone/3.0.17-x86_64-ruby1.9.3-linux-gcc4.4.6-1002/support/lib/phusion_passenger/rack/application_spawner.rb 129 in `start'
20  /var/lib/passenger-standalone/3.0.17-x86_64-ruby1.9.3-linux-gcc4.4.6-1002/support/lib/phusion_passenger/spawn_manager_orig.rb   253 in `block (2 levels) in spawn_rack_application'
21  /var/lib/passenger-standalone/3.0.17-x86_64-ruby1.9.3-linux-gcc4.4.6-1002/support/lib/phusion_passenger/abstract_server_collection.rb   132 in `lookup_or_add'
22  /var/lib/passenger-standalone/3.0.17-x86_64-ruby1.9.3-linux-gcc4.4.6-1002/support/lib/phusion_passenger/spawn_manager_orig.rb   246 in `block in spawn_rack_application'
23  /var/lib/passenger-standalone/3.0.17-x86_64-ruby1.9.3-linux-gcc4.4.6-1002/support/lib/phusion_passenger/abstract_server_collection.rb   82  in `block in synchronize'
24      prelude>    10:in `synchronize'
25  /var/lib/passenger-standalone/3.0.17-x86_64-ruby1.9.3-linux-gcc4.4.6-1002/support/lib/phusion_passenger/abstract_server_collection.rb   79  in `synchronize'
26  /var/lib/passenger-standalone/3.0.17-x86_64-ruby1.9.3-linux-gcc4.4.6-1002/support/lib/phusion_passenger/spawn_manager_orig.rb   244 in `spawn_rack_application'
27  /var/lib/passenger-standalone/3.0.17-x86_64-ruby1.9.3-linux-gcc4.4.6-1002/support/lib/phusion_passenger/spawn_manager_orig.rb   137 in `spawn_application'
28  /var/lib/passenger-standalone/3.0.17-x86_64-ruby1.9.3-linux-gcc4.4.6-1002/support/lib/phusion_passenger/spawn_manager.rb    16  in `spawn_application_with_env'
29  /var/lib/passenger-standalone/3.0.17-x86_64-ruby1.9.3-linux-gcc4.4.6-1002/support/lib/phusion_passenger/spawn_manager_orig.rb   275 in `handle_spawn_application'
30  /var/lib/passenger-standalone/3.0.17-x86_64-ruby1.9.3-linux-gcc4.4.6-1002/support/lib/phusion_passenger/abstract_server.rb  357 in `server_main_loop'
31  /var/lib/passenger-standalone/3.0.17-x86_64-ruby1.9.3-linux-gcc4.4.6-1002/support/lib/phusion_passenger/abstract_server.rb  206 in `start_synchronously'
32  /var/lib/passenger-standalone/3.0.17-x86_64-ruby1.9.3-linux-gcc4.4.6-1002/support/helper-scripts/passenger-spawn-server

UPDATE more info: I decided to log into the instance with ec2-user and notice that git is installed the gems I need ARE installed so far I can see.. when I do bundle list they all show up even the one that the error is complaining about.. I do notice that when I do a general gem list then it DOES not show up... but it should be using the bundle one.

cfernandezlinux
  • 861
  • 1
  • 10
  • 23

5 Answers5

57

(Note that the following workaround should only be used if you must use Git sources for dependencies. It is recommended not to install dependencies from external Git repositories if it can be avoided. See below for details on why that is.)

When using Git backed libraries in a Gemfile with Passenger, you must disable shared gems in an installation (in addition to installing Git in the hook you listed above). You can do this by setting the BUNDLE_DISABLE_SHARED_GEMS Bundler environment variable in your existing .ebextensions/ruby.config file like so:

option_settings:
  - option_name: BUNDLE_DISABLE_SHARED_GEMS
    value: "1"
  - option_name: BUNDLE_PATH
    value: "vendor/bundle"

packages:
  yum:
    git: []

Disabling shared gems will force all dependencies to be vendored into your application in vendor/bundle as specified by the BUNDLE_PATH variable.

Note that, whenever possible, you should avoid installing public libraries from Git sources with your application. Using Git for library locations introduces another point of failure for a deployment install, since the Git repository may be temporarily unavailable or even permanently moved. Also keep in mind that forcing vendored installs in a deployment will cause your Elastic Beanstalk deployments to be much slower on subsequent deploys of an app with the same dependencies. This is because the libraries will be re-installed at each deploy instead of taking advantage of the system-wide installation that Elastic Beanstalk has Bundler perform by default.

In short, if there is an official RubyGem release of the library in question, you should use that version instead; and if not, you should suggest to the library author that an official RubyGem release be made available.

FYI a similar question about this Git problem with regular Passenger/Rails deployments was previously asked: Rails 3: Passenger can't find git gems installed by bundler

Community
  • 1
  • 1
Loren Segal
  • 3,251
  • 1
  • 28
  • 29
  • so then the hook/EB will install everything to vendor/bundle for you? so no need for me to send a hook with the bundle package command right? – cfernandezlinux Dec 02 '12 at 02:10
  • 1
    hmm weird.. I try to do it your way and it does create the vendor/bundle but I go back to having the missing gem issue.. :-( – cfernandezlinux Dec 02 '12 at 02:24
  • I'm unable to reproduce this. I created a completely fresh application with the refinerycms and refinerycms-search gems and it was able to load them with the above ebextension. Have you tried on a fresh container? Also, is it still the same gem that is failing? – Loren Segal Dec 04 '12 at 03:49
  • I did what suggested in the above answer and everything worked like a charm, except when a new instance has been launched and failed to retrieve gem using git. – matteo Jan 08 '13 at 14:42
  • 5
    @LorenSegal rather than bundling shared gems, you can get the same behavior (not reinstalling all gems on every deploy) *without* breaking git gems by symlinking vendor/bundle to a stable location and then running `$ bundle --deployment`. See gembundler.com/v1.2/rationale.html under Deploying Your Application. Also, there are several good reasons to bundle from git; many folks that arrive at this question are bundling code from git that is not on RubyGems. – Gabe Kopley May 18 '13 at 23:42
  • 2
    FWIW, setting the `BUNDLE_*` env vars doesn't seem necessary when running in puma (as opposed to passenger). – Jon Burgess Feb 11 '16 at 00:56
14

Another option is to package the gem source directly with your application and then point bundler at that.

Copy the gem source into vendor/gems/mygem

Then, in your Gemfile:

gem 'mygem', path: File.join(File.dirname(__FILE__), 'vendor', 'gems', 'mygem')

See more here: http://viget.com/extend/bundler-best-practices

Micah
  • 17,584
  • 8
  • 40
  • 46
  • Since I use Puma and not Passenger, I decided to go this route and it worked fine. For the ones wondering (like I was), I just licked "download as zip" in Github, and pasted the entire folder in /vendor/gems/#{gem_name} and it worked. – sandre89 May 01 '17 at 15:12
6

Amazon's Elastic Beanstalk Ruby AMI needs a little tweaking in order to allow you to bundle gems from git without sacrificing deployment speed, behavior you get out of the box with Capistrano and Heroku.

Fortunately, the Elastic Beanstalk configuration API makes the necessary tweaks possible without requiring you to maintain a custom AMI.

Here's the Elastic Beanstalk configuration that I use to get the desired, conventional Ruby deployment behavior with Amazon's own AMI: https://github.com/gkop/elastic-beanstalk-ruby .

Gabe Kopley
  • 16,281
  • 5
  • 47
  • 60
  • This file dosent do db:create? When does that happen? – Pinser Jun 10 '14 at 15:06
  • @DeBuGGeR other rails deployment systems like Capistrano and Heroku do not typically run `rake db:create` either. In the case of Elastic Beanstalk, the `eb init` command gives you an option for creating an RDS database. I haven't personally tried it, but this blog post appears to be a good starting point: https://medium.com/ruby-on-rails/how-to-set-up-a-rails-4-1-app-on-aws-with-elastic-beanstalk-and-postgresql-66d4e3412629 – Gabe Kopley Jun 10 '14 at 17:35
  • so when does db:create occurs? eb init also did not run db:create or db:migrate or db:seed. And my application is showing a 502, i m lost no one seems to have addressed or explained the issue anywhere. I have finished AWS docs as well. – Pinser Jun 11 '14 at 07:54
  • I believe you don't need to run `rake db:create` since `eb init` creates the RDS db for you. Elastic beanstalk runs `rake db:migrate` automatically. If you want `rake db:seed` to run I believe you have to add it to your config in `.ebextensions/`. – Gabe Kopley Jun 11 '14 at 17:22
  • Well, thanks for resolving this I connected directly to Amazon RDS and executed this. But you are right! Thanks! – Pinser Jun 12 '14 at 07:57
5

After trying the accepted answer, I found that a simpler .ebextensions/ruby.config was the only config that worked:

packages:
  yum:
    git: []
Justin Tanner
  • 14,062
  • 17
  • 82
  • 103
  • 1
    Same for me, except I had to rename the file to `.ebextensions/00_packages.config`. Otherwise git seemed to be installed too late. The file name is critical apparently – Steve Potter Mar 22 '18 at 16:26
  • I actually had to name the file ruby.config as mentioned above. Otherwise it would install git too late to install the required gems. – Fabio Jul 10 '21 at 13:04
2
  • ok well after lot of research and testing I figure this was related to Amazon having some issues with the passenger env values...

  • I was able to run manually rails s and then it worked fine and all gems loaded.. but with passenger it did not so find that if I run

    bundle pack --all

  • --all so it also packs the git gems..

  • it will then run like a charm..

  • to be able to have this run in the mean time Amazon fixes this issue with bundle I create a hook and force it to run after every install. not the best way but works.

Note: using the hook I cant seen to run the command on EB updates so I git add the vendor/cache and it uploads it all by default.

cfernandezlinux
  • 861
  • 1
  • 10
  • 23
  • I'm trying this solution, vendoring all git gems. Since with the approach described in the previous answer, as soon as a new instance was launched it failed on installing the required gems from git – matteo Jan 08 '13 at 14:46