How we use environments in Chef at Recorded Future

A couple of months ago Opscode introduced the environment concept in Chef, a long awaited feature. Still not perfect but a very useful feature. At Recorded Future are we using subversion as our versions management system and trying to keep the work in the trunk but from time to time we alsohave one or two feature branches, and until we are in the trunk only heaven (read more about why avoid feature branches in this blogpost by Jez Humble) we have to cope with managing branches for different Chef environments.

Today we have different environments for different trunk and branches but also for different setup for our system, we have our main production system but we are also managing our system for specific customers at separate machines with separate configurations. We also use green-blue deployment that means for some time we live with two environments on released and one to release.

An example how we a setup of environments could look like this:

Chef environment management

A cookbook in Chef could have a version number that has three parts like 1.0.1 major, minor and patch version. We are using are using different major number for the trunk and the different branches. So cookbooks in trunk starts with 12.x.x and in a branch they could start with 13.x.x, and after we have merged our branch back into trunk then the cookbooks in trunk will start with 13.x.x and the next branch will have 14.x.x as major version number. Then we use the second number to determine which environment within the trunk/or branch that the cookbook is valid for

An example of our environments and version used in the cookbook for the different environment

Env Used for From Ver
r12a Released main production system Chef recipes from trunk a specific revision 12.0.x
r12b Main production system to be released Chef recipes from trunk – head – latest version in trunk 12.1.x
cust_a A system for a customer Chef recipes from trunk a specific revision)/ 12.2.x
r12test A test environment for manual tests 12.98.x
r12build A test environment for unit, integration and systems test used in our build process Chef recipes from trunk – head 12.99.x
r13a A branch to be released Chef recipes from a branch – a specific version 13.0.x
r13build A test environment for unit, integration and systems test used in our build process Chef recipes from a branch – head 13.99.x

Then in Chef we set the version of the cookbook in the different environments with the version constraint ~> for example rfcommon ~> 12.0.0 for r12a, we do this with a script that generate an environment specification file for all cookbooks that we upload to the Chef server for each environment.

But now we have the problem how to maintain the version number for the cookbook in the metadata.rb file. We have earlier created a script (no source published yet) for uploading cookbooks to save some typing and that do some check if the cookbook is checked-in in subversion etc and then the script does a knife cookbook upload command. We started with the script before version 0.10 of knife when Opscode introduced the plugin concept, so maybe we will later create a knife plugin instead.

Now we also have extended our script to handle the version management for the cookbooks. We use the script with an environment parameter eq r12a then the script temporarily change in the metadata.rb file for the cookbook to be uploaded. The script set the correct version number according for the environment parameter by replacing major and minor version number with the version number for the environment. Eq 12.0 for environment r12 but the script doesn’t touch the last number the revision number. After the cookbook is uploaded the script changes back to the original metadata.rb file, this to avoid a lot of mismatches during merge of branches etc.

The script will also validate that the path given to the script match the branch/trunk used for the environment to avoid uploading code from wrong branch/trunk to an environment

Advertisements

3 comments so far

  1. Dan Nemec (@dcnem) on

    Good to hear someone else is using Chef Environments for Product release versioning. Their documentation gives the use-case of “dev” “qa” and “prod” for environments, but we make all our environments look almost identical. The variance is in the product release. We version our cookbooks with our product and set an environment per release version. It is very similar to what you do.

  2. ulfmansson on

    Yes, I think the original design was for environments like test,qa and prod, but with some enhancements it would work better for other type of versioning/environments.

    I would like to be able to not store the version number in the Metadata.rb file and least be able to set it dynamically when you run the knife cookbook upload command.

  3. plumber in london on

    Having read this I thought it was rather informative.
    I appreciate you taking the time and energy to
    put this article together. I once again find myself personally spending way too much time
    both reading and leaving comments. But so what, it was still
    worthwhile!


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: