Manage configuration files across environments

Question

How do you (your company) manage the config-files of the apps/systems you build? Let me tell you how we do it, and what the problem is.

I'm working at a company where we develop software with about 15 developers. We build line-of-business web apps that are deployed at our managed hosting provider. One of our main apps consists of one web site and about ten WCF services. Some of the services are connected to each other.

I don't know if this is a big system, or small, but my opinion is that is takes us way too long to get things up and running in our different environments (test, acceptance and production).

We have config-files per environment in our Visual Studio projects. So a web.test.config, a web.acc.config, a web.prod.config and a web.config for development. They all have the same keys in them, but the values can be different, depending on the environment they are meant for.

If I do a quick count of the appsettings in the web.config for the webapp I count 32. And I count 5 endpoints. We have four environments (dev, test, acc and prod) this means 128 appsettings and 20 endpoints in total for one web app. We can easily make mistakes, especially when deadlines are closing in.

We are all humans, so things like this may happen to anyone:

• We make a change in one of config files, but forget to check in before we build and deploy.
• Or we make a change on the WebServer and forget to update accordingly in four other web.configs.
• Or we Change only three of four config files. And so on.

Then we have the infrastructure at our managed hosting provider. By default every port is closed. So if one of WCF services needs to talk to other one of WCF services located on a different server, a firewall protected port has to be open.

We do this in Test, but in Acceptance we have to do it again, and we have forgotten which ports have to be opened, so it's more like trial-and-error: Oh my service can't connect to the database, probably the port is closed. The same problem may happen in production as well.

Our managed hosting provider can take a few days to open a port in a firewall, according to the SLA. So, this quickly becomes a pretty long process. And in the end it takes us about two months to have Test, Acceptance and production up and running.

So, my question is: how do you manage the configurations and the infrastructure and the process around it?

Answer 1

The Config4* project (disclaimer: I am its primary developer) does not have an out-of-the-box integration with .Net or WCF, so it is probably not useful to you. However, one of the features in Config4* is relevant to your question: it is the ability to embed if-then-else statements in a configuration file, so that the file can "adapt" itself for different environments (such as development, testing, acceptance and production).

You may be able to modify that concept to work with whatever configuration syntax you are using in your .Net/WCF-based project (I'm not familiar with those technologies, but I'm guessing they probably use XML-based configuration files). In particular, you could write a script using, say, Python, that uses if-then-else statements to set environment-specific name=value pairs in a map, and then use some print statements to generate a set of configuration files tailored for an environment. A pseudo-code outline of such a script is:

#--------
# Set up configuration variables suitable for a specified environment
#--------
cfg["variable1"] = "default value";
cfg["variable2"] = "another default value";
if (environment == "testing") {
  cfg["variable1"] = "override default value for this environment";
  cfg["variable3"] = "value suitable for this environment";
  ...
} else if (environment == "production") {
  ...
}
#--------
# Now use print statements to generate configuration files
# Alternatively, use the _name=value_ pairs in the map to
# perform global search-and-replace on template versions of
# configuration files.
#--------
...

For bonus points, the script could also generate a checklist of tests that need to be performed for the environment, for example, "Check if a firewall port needs to be opened between the following endpoints: ..."