Example 42 Puppet Infrastructure How To

This page is outdated.

Lab42's proposal of recipes and a coherent infrastructure is a work in progress.
Currently the Example 42 Puppet Infrastructure can be considered at alpha stage:
- Must be tested with general settings
- Modules clean up and standardization is needed
- Some dependencies should be fixed
- Works and is tested on Centos/Redhat 5 (should behave well on all Redhat based distros)
- Suse / Debian support is just planned but not fully implemented

It works and has been tested on Puppet 0.24.7.
Earlier version can give (easily fixable) errors in some modules.

It's made to be easily adapted to custom needs (yours).

Note, easy doesn't mean unaware. You need to know few basic principles in order to customize at best.

Step 1 - Download and "install"

You can browse it here or you can download it via anonymous git:
git clone git://www.example42.com/example42puppetinfrastructure
The repository contains different modules (recipes) each in its own directory. You can place them in the default modules dir of your puppetmaster ( /var/lib/puppet/modules ) or in a custom directory ( a suggested path (since it's expected here) is /etc/pupppet/data/yourprojectname/ ).

So, if your project name is "acme" (you can use the name of your company, of your customer, of a specific project within your company... whatever you find suitable as your project name):

yum install puppet puppet-server
mkdir -p /etc/puppet/data/acme
cd /etc/puppet/data/acme
git clone git://www.example42.com/example42puppetinfrastructure .

Step 2 - Configure Puppet

Once downloaded the "source" Example42 modules you need to configure your puppet master to use them.
More precisely you must define the modulepath you've used (if not the default), your puppet master name and convert the set of example42 modules to your project.
In this case we use "acme" as your project name, and puppet.acme.com as the FQDN of your puppet master (should be also it's local hostname to avoid problems with certificates).
A simple script to convert the sample project_example42 in you custom project_acme is provided here, just edit it and change the value of the NEWPROJECT variable.

echo "modulepath = /etc/puppet/data/acme:/var/lib/puppet/modules" >> /etc/puppet/puppet.conf
echo "server = puppet.acme.com" >> /etc/puppet/puppet.conf
cd /etc/puppet/data/acme/
vi example42_project_clone.sh # (set NEWPROJECT="acme")
sh example42_project_clone.sh

When you execute the script you create a project_acme directory cloned from the existing project_example42 directory, where you can define all the nodes of you infrastructure. All the references to "example42" (file names and string contained in files) are converted to "acme".

Finally you have to edit the default startup manifest to point to your project class:

vi /etc/puppet/manifests/site.pp # (here you just need to write a line with "import project_acme")

Step 3 - Customize

Now you can start to customize your manifests in order to reflect your infrastructure logic.
First of all note that project_acme (or project_exaample42 or project_whatever) is just a module that defines your infrastructure, you can completely change and adapt it to your needs.
The other modules, more or less application oriented, are where real configurations are defined, generally these are simply included in the nodes defined in project_acme. Each of these modules is organized according to Reductive Labs suggested Module organization.
In module_name/manifests/init.pp there is the starting manifest.
Here you find a class (with the same name of the module) and possible subclasses for specific functions of the same applications (ex: cobbler and cobbler::full).
The primary class (ex: samba) should not overwrite configuration files but only define the general types, whose properties can be modified by subclasses that inherit the main one (ex: samba::pcd)

The project_ module needs some explanation.
Let's see how the proposed project_example42 is organized, but always keep in mind that the organization of this "structural" module, which is also the first module to be included in puppet, can be totally changed.

init.pp is the first manifest parsed by Puppet when you include project_example42, this just sets up some site-wide default settings and imports other manifests

infrastructure.pp defines your overall infrastructure logic. Here different zones are defined (it's good idea to consider a zone as a dedicated IP network or as a specific functional area (ex: dmz1, dmz2, frontend, backend, intranet, devel etc)) in order to distinguish nodes according to different zones. For more info about the logic of zones and roles give a look to InfrastructureDesignGuidelines. Here various variables (with prefix my_) are used in order to define settings used by the modules included

nodes.pp defines your specific nodes settings. Here you MUST change the node names according to your puppet client hostnames. A single host/node can inherit existing "zone/nodes" with the possibility of overriding variables defined at more general levels. You CAN decide if it's better for your case to include different modules directly at node level, or include more general roles, that can be used by more nodes. In project_example42 are used these custom roles, so at node level you just include a specific role.

roles.pp define the various roles used by the nodes. There is a general class with "general" modules used in all the hosts and then some sample specific roles with extra includes.
It's a good idea to define the variable $my_role with the name of the role and $my_zone with the name of the zone (so you can differentiate configurations according to the node's role or zone).

Note that according to this design a node inherits a zone and includes a role:
- Only ONE inheritance tree for nodes: general node - [zone/network/environment] - [ subzone ] - node
- Each node may include a SINGLE specific role, where you define what modules to use

Now, how to organize your own infrastructure actually depends only on its complexity and specificity.
What must be respected is the use of few specific custom variables:
- $my_project defines the name of the project. You can have different projects defined in different project_projectname modules that share the common modules.
- $my_zone defines the zone (environment) inherited by the host node (devel, test, prod..),
- $my_role defines the role of the host node (webserver, proxy, mail... )

Other variables might be requested by specific modules, for example:
- $my_update defines if packages must be automatically updates (yes/no becomes ensure => latest/present in packages definitions),
- $my_puppet_server defines the servername of the puppet master (used in the puppet module),
- $my_ntp_server defines the servername of the NTP server (used in the ntp module),
- $my_syslog_servers defines the servername of the syslog server(s) (may be an array) (used in the syslog module),
You can decide to apply and use these variables according to the modules you decide to use.
A fast way to see what module uses what variable (for example "$my_network" is a command like this at the root of your modules directory

grep -R my_network *

use similar searches to reconstruct the logic of the Example 42 Puppet Infrastructure and realize how its components are assembled.