Archive for the ‘Chef’ Category

How we use environments in Chef at Recorded Future

A couple of months ago Opscode introduced the environment concept in Chef, a long awaited feature. Still not perfect but a very useful feature. At Recorded Future are we using subversion as our versions management system and trying to keep the work in the trunk but from time to time we alsohave one or two feature branches, and until we are in the trunk only heaven (read more about why avoid feature branches in this blogpost by Jez Humble) we have to cope with managing branches for different Chef environments.

Today we have different environments for different trunk and branches but also for different setup for our system, we have our main production system but we are also managing our system for specific customers at separate machines with separate configurations. We also use green-blue deployment that means for some time we live with two environments on released and one to release.

An example how we a setup of environments could look like this:

Chef environment management

A cookbook in Chef could have a version number that has three parts like 1.0.1 major, minor and patch version. We are using are using different major number for the trunk and the different branches. So cookbooks in trunk starts with 12.x.x and in a branch they could start with 13.x.x, and after we have merged our branch back into trunk then the cookbooks in trunk will start with 13.x.x and the next branch will have 14.x.x as major version number. Then we use the second number to determine which environment within the trunk/or branch that the cookbook is valid for

An example of our environments and version used in the cookbook for the different environment

Env Used for From Ver
r12a Released main production system Chef recipes from trunk a specific revision 12.0.x
r12b Main production system to be released Chef recipes from trunk – head – latest version in trunk 12.1.x
cust_a A system for a customer Chef recipes from trunk a specific revision)/ 12.2.x
r12test A test environment for manual tests 12.98.x
r12build A test environment for unit, integration and systems test used in our build process Chef recipes from trunk – head 12.99.x
r13a A branch to be released Chef recipes from a branch – a specific version 13.0.x
r13build A test environment for unit, integration and systems test used in our build process Chef recipes from a branch – head 13.99.x

Then in Chef we set the version of the cookbook in the different environments with the version constraint ~> for example rfcommon ~> 12.0.0 for r12a, we do this with a script that generate an environment specification file for all cookbooks that we upload to the Chef server for each environment.

But now we have the problem how to maintain the version number for the cookbook in the metadata.rb file. We have earlier created a script (no source published yet) for uploading cookbooks to save some typing and that do some check if the cookbook is checked-in in subversion etc and then the script does a knife cookbook upload command. We started with the script before version 0.10 of knife when Opscode introduced the plugin concept, so maybe we will later create a knife plugin instead.

Now we also have extended our script to handle the version management for the cookbooks. We use the script with an environment parameter eq r12a then the script temporarily change in the metadata.rb file for the cookbook to be uploaded. The script set the correct version number according for the environment parameter by replacing major and minor version number with the version number for the environment. Eq 12.0 for environment r12 but the script doesn’t touch the last number the revision number. After the cookbook is uploaded the script changes back to the original metadata.rb file, this to avoid a lot of mismatches during merge of branches etc.

The script will also validate that the path given to the script match the branch/trunk used for the environment to avoid uploading code from wrong branch/trunk to an environment