Installing a puppet module from a manifest script

Question

I'm using puppet to provision a vagrant (ubuntu based) virtual machine. In my script I need to:

sudo apt-get build-dep python-lxml

I know I can install the apt puppet module so I can use:

apt::builddep { 'python-lxml': }

But I can't find any reference about installing a module from the script and how to include/require it. Seems to me that the puppet docs refer only to installing from the command line puppet tool

I also tried doing something like:

define build_dep($pkgname){
    exec {
    "builddepend_$pkgname":
    commmand => "sudo apt-get build-dep $pkgname";
    }
}
build_dep{
    "python-imaging":
    pkgname => "python-imaging";

    "python-lxml":
    pkgname => "python-lxml";
}

But puppet exited with an error on this. And also:

exec{"install apt module":

    command => "puppet module install puppetlabs/apt"
}

class { 'apt':
        require => Exec["install apt module"]}
include apt

apt::builddep { 'python-imaging':
 }

but got could not find declared class apt at..

any ideas? directions? I know I'm missing something obvious but can't figure this out.

EDIT: If I pre-install (with puppet module install from the commandline) the apt:builddep works fine. But I need puppet to handle the module downloading and installation. Some of the other work arounds also work for the basic use case but won't answer my main question.

Answer 1

I ran into this problem as well. The trick is to download the modules using a vagrant shell command before the puppet provisioner runs.

config.vm.provision :shell do |shell|
  shell.inline = "mkdir -p /etc/puppet/modules;
                  puppet module install puppetlabs/nodejs;
                  puppet module install puppetlabs/apache"
end

config.vm.provision :puppet do |puppet|
  puppet.manifests_path = "puppet/manifests"
  puppet.manifest_file = "site.pp"
end

Order is important here, and since the puppet provisioner hasn't run the folder /etc/puppet/modules does not exist yet.

The reason I decided, like alonisser, to install the modules using the puppet module tool instead of using a module folder with the vagrant puppet provisioner was because I didn't want to have to download all of the dependencies of the modules I was going to use and store all of those modules in my source control. Running these two commands results in 5 dependencies that would otherwise sit in my git repository taking up space.

Answer 2

Here's what I did to make the puppet module install command run at most once:

$script = <<EOF
mkdir -p /etc/puppet/modules
(puppet module list | grep puppetlabs-mysql) ||
   puppet module install -v 2.1.0 puppetlabs/mysql
EOF

Vagrant::Config.run do |config|
   config.vm.provision :shell, :inline => $script

Answer 3

I use an approach similar to @brain_bacon - the additional complication I had is that in addition of pre-packaged modules like puppetlabs/nodejs I needed local modules relative to my Vagrantfile. I didn't want to check in the pre-packaged modules as part of my repository, nor use git submodules because of the problems pointed out by @Igor Popov.

Finally the solution I chose was to use a shell script to download the modules, but forcing their path into the shared directory between the Vagrant VM and the host, and using .gitignore to avoid that path being under source control.

To be clear, with this tree:

jcmendez$ tree
.
├── README.md
├── Vagrantfile
├── files
├── puppet
│   ├── manifests
│   │   └── init.pp
│   └── modules
│       ├── misc
│       │   └── manifests
│       │       └── init.pp
│       ├── mysql
   ...
│       └── wordpress
│           ├── files
│           │   ├── wordpress-db.sql
│           │   ├── wp-config.php
│           │   └── wp-tests-config.php
│           └── manifests
│               └── init.pp
└── wordpress

On .gitignore I added

  /puppet/modules/mysql

On Vagrantfile

  config.vm.provision :shell do |shell|
    shell.inline = "puppet module install puppetlabs/mysql --force --modulepath '/vagrant/puppet/modules'"
  end

  config.vm.provision :puppet do |puppet|
    puppet.manifests_path = 'puppet/manifests'
    puppet.module_path = 'puppet/modules'
    puppet.manifest_file  = "init.pp"
    puppet.options="--verbose --debug"
  end

Answer 4

You can install puppet modules on the host machine within your vagrantdir:

puppet module --modulepath=./puppet/modules/ install puppetlabs/mysql

Vagrant takes care to mount this directory to the right place. So there is no need to run scripts other than puppet itself on the node.

Answer 5

Inspired by wtanaka.com's answer, I implemented a solution like below which I think is much more readable.

1) create a new ruby file called 'puppet_deps.rb' under the same folder as Vagrantfile with code like below:

def install_dep(name, version, install_dir = nil)
    install_dir ||= '/etc/puppet/modules'
    "mkdir -p #{install_dir} && (puppet module list | grep #{name}) || puppet module install -v #{version} #{name}"
end

2) In your Vagrantfile, you can load this ruby file and use it to specify the puppet dependency:

# on top of your Vagrantfile
require './puppet_deps'
...
...
# in your vm definition, use a shell provisioning this:
config.vm.provision :shell, :inline => install_dep('puppetlabs-firewall', '1.1.3')
config.vm.provision :shell, :inline => install_dep('puppetlabs-stdlib', '4.3.2')

Answer 6

@brain_bacon's strategy worked almost perfectly for me, with one small caveat - if the module already exists, then the provisioning script fails, halting the provisioning process. The following adjustment fixed this:

config.vm.provision :shell, :run => "always" do |shell|
  shell.inline = %{
    mkdir -p /etc/puppet/modules;
    function install_module {
      folder=`echo $1 | sed s/.*-//`
      if [ ! -d /etc/puppet/modules/$folder ]; then
        puppet module install $1
      fi
    }
    install_module stdlib
    install_module apt
    install_module ruby
  }
end

Answer 7

You can create a directory for modules and add the apt module that you downloaded from the forge. So it will be modules/apt/*. Then you can specify this module directory in Vagrantfile (module_path is relative to Vagrantfile):

Vagrant.configure("2") do |config|
  config.vm.provision :puppet do |puppet|
    puppet.module_path = "modules"
  end
end

For more information see the documentation.

Answer 8

Using answers from this question I created this script:

#!/bin/bash

function install_module {
   IFS=':' read module version <<< "$1"
   if (puppet module list | grep $module ) >/dev/null; then
    echo "Module $module is already installed"
   else
        if [ -z "$version" ]; then
            puppet module install $module
        else 
        puppet module install -v $version $module
    fi
    fi  
}

if dpkg --compare-versions `puppet --version` "lt" 3.8.7; then
    sudo apt-get remove --purge -y puppet
    sudo apt-get -y autoremove
fi

if which puppet >/dev/null; then
    echo "Puppet is already installed"
else
    echo "Installing puppet"
    wget https://apt.puppetlabs.com/puppetlabs-release-trusty.deb
    sudo dpkg -i puppetlabs-release-trusty.deb
    sudo apt-get update
    sudo apt-get install -y puppet=3.8.7*
    mkdir -p /etc/puppet/modules
fi

for var in "$@" 
do
    install_module "$var"
done

Then you can use it from your Vagrantfile like this:

puppet_modules = [
    'puppetlabs-stdlib:4.16.0',
    'puppetlabs-apt:2.3.0',
    'puppet-nodejs:2.3.0'
    ]

config.vm.provision "shell", path: "puppet/scripts/puppet.sh", args: puppet_modules.join(" ")

It removes puppet from vm if version is lower than 3.8.7, then installs puppet 3.8.7, and then installs all the modules.

It would probably work only on debian/ubuntu boxes.