0

I recently completed and deployed a personal project which turned out to require rather more computing power than I’d originally estimated. To fuel this demand, I opened a Rackspace cloud account to host my development and production environments and began configuring my environment as I wanted it. If you haven’t used Rackspace cloud before, I can highly recommend it – it’s a fairly basic Cloud Service which is built upon OpenStack and besides providing great quality nodes with a minimum of fuss, is reasonably priced. You also could also use your newly created puppet manifests to work with a system like vagrant if you didn’t wish to work with a hosted service in development.

Being an advocate of Continuous Delivery, I wanted to setup a pipeline for my software and I needed a consistent development and production environment and I didn’t like the idea of the repetitive configuration process every time I wanted to shut down, start up or create a new node. It isn’t too much effort to manually configure a single node but as I was being charged for my compute usage whilst the systems were running, why would I want to keep them running when I didn’t actually need them? The other Continuous Delivery principle at work here is that or Automating everything; after all, a manual process can never be described as reliably repeatable.

To this end, there are two main competing open-source automation systems in this space: Opscode Chef and Puppet. Both are primarily written in and reliant upon Ruby (and also annoyingly difficult to effectively google!). The primary different between them is that:

  • Chef describes systems in an imperative fashion.
  • Puppet describes systems in a declarative fashion.

I have previously experimented with Chef so decided to setup and configure my systems with puppet. It is worth reading the puppet introduction as well as the tutorials available on puppet labs.

I opted to use puppet masterless as I didn’t want to maintain a puppetmater instance. My second iteration of my Rackspace cloud infrastructure will include a puppetmaster instance, but I always feel you should learn to walk before you run!

First, I installed puppet onto my Ubuntu node. In this instance I will opt to install puppet and then create a handy snapshot, which I will be able to clone later to add a pre-puppet client’d node. If I can create a snapshot then why not just create a snapshot of my configured system and use that? The short answer is that snapshots are harder to maintain if you’re tweaking and re-configuring them.

I then created a directory with two sub directories:

-- jamiei-puppet
--- manifests
---- site.pp
--- modules

site.pp is where our node description lies. This is our puppet manifest, a basic unit of execution which can reference resources, other manifests and generally manage aspects or our system. A basic manifest, creating a single file, might look like this:

    file {'helloworld':
      path    => '/tmp/helloworld',
      ensure  => present,
      mode    => 0640,
      content => "Hello, world",
    }

We can test this first by moving to our jamiei-puppet directory and running:

puppet apply --noop manifests/example.pp

The –noop flag tells puppet to not actually make the changes, so my output looks something like this:

notice: /Stage[main]//File[helloworld]/ensure: is absent, should be present (noop)

Nicely informing me that it couldn’t find our file and so it would have created it.

We might only want some of our nodes to be decorated with our magical greeting message but not to duplicate our code across lots of different roles. Puppet manifests can contain what are essentially inherited roles:

node base_jamiei_box {
}

node default inherits base_jamiei_box {
}

node 'puppet-test.jamiei.com' inherits default {
    file {'helloworld':
      path    => '/tmp/helloworld',
      ensure  => present,
      mode    => 0640,
      content => "Hello, world",
    }
}

As you might be able to see from the above, there is an inheritance chain which allows me to layer my roles for a particular box accordingly, culminating with our helloworld file being applied only to this node only.

Puppet allows blocks of resource manipulation to be bundled into modules. For example, I quite like my machines to be provisioned with ntp so that my servers are all synchronised. To this end, I could include an ntp module in my default role. To do this, I created a subdirectory in the modules folder called ntp, with another subfolder called manifests and create a file called site.pp within it (site.pp is a convention). This file contains my class name and the declarations for ensuring that the ntp package is installed, before ensuring that the service is running:

class ntp {
  package { "ntp": ensure => installed }
  service { "ntpd": ensure => running }
}

If the package isn’t installed, it will be and if the service isn’t running then it will be started. I can then include this into a node’s configuration by simply include ntp ‘ing. In this way, I can build up my nodes configuration. I can inspect puppet’s view of my node via the puppet resource command. For example, the manifest below creates me a user on the box and then adds my authorised key to the box so that I can login.

node base_jamiei_box {
    include ntp
    include base-dev-packages
}

node default inherits base_jamiei_box {

  user { 'jamie':
    shell => '/bin/bash',
    uid => '1000',
    ensure => 'present',
    gid => '1000',
    groups => ['sudo'],
    home => '/home/jamie',
    comment => 'Jamie,-,-,-,-'
  }
  ssh_authorized_key { "jamie":
    ensure => present,
    key => "INSERT PUBLIC KEY HERE",
    name => "no@spam.com",
    user => 'jamie',
    type => rsa,
  }

}

node 'puppet-test.jamiei.com' inherits default {
    file {'helloworld':
      path    => '/tmp/helloworld',
      ensure  => present,
      mode    => 0640,
      content => "Hello, world",
    }
}

It’s never particularly sensible to re-invent the wheel needlessly and it should be no great surprise that there is a strong community which have created modules for almost every conceivable task. You can (and probably should) make use of one of the many puppet modules which are available on the web. For my own project, I made use of these modules:

You can include a top level root module directory in masterless mode by using the –modulepath command line option:

puppet apply --noop --modulepath modules manifests/example.pp

It goes without saying that your puppet manifests should be versioned, I prefer git for this purpose. It should be fairly apparent to you now how puppet might ensure that your environment is consistent and also to automatically fashion a new node into your preferred state by simply cloning my masterless recipe and then running puppet apply.

Further Reading

Tags: , ,

Leave a Comment