Configuration Rodeo
March 21, 2012
Configuration Rodeo
The history of configuration file conventions for Ruby projects has not been a particularly conclusive one. No one pervasive convention has ever developed. No developer has ever laid down a clear set of rules of engagement that we all can agree are worthy of follow. Rather, the realm of the developer in working out how to handle configuration for their project tools still smacks a bit of the Wild West.
Nonetheless, although no one approach has superseded all others, there are handful of patterns that tend to be used. We will review these conventions and then consider a possible comprehensive solution that could move us past the chaos du jour.
The Conventions
1. Dot File
Probably the oldest convention around is the dot-file. A dot-file is, as the name implies, a file with a dot in front of its name. Using traditional Unix shells, files that begin with a dot are hidden files. For the most part all this means is they do not show up in the typical directory listing, although some file tools will also ignore dot-files by default.
Back in the day, these files became the conventional means of adding per-user configuration to a system by adding dot-files to a user’s home directory. There it was easily associated to a user and accessible, yet hidden from normal view. Consequently, project tools have adopted the same idea for project configuration, evidenced most clearly by SCM tools like Subversion (.svn), Git (.git) and Mercurial (.hg). Many Ruby-based tools have followed suite, such as RDoc’s .document and YARD’s .yardopts and .yardoc.
The advantages of the dot-file lies mostly in the fact that it’s a well worn standard, and that it helps somewhat with directory clutter by keeping these files hidden under a typical listing. However, for the same reasons they are also a disadvantage. Many types of file listings do not actually hide these files, for example a GitHub listing, nor is it always advantageous that they be hidden because an important configuration file could go accidentally overlooked. Also dot-files are often without a proper file extension. While file extensions feel archaic, the fact remains that file extensions are still the primary way that file types are identified.
2. Crapfile
The so called “Crapfile” convention is another pattern that has been around for a long time in the form of the rather ubiquitous Makefile. Almost all C/C++ projects use the Make tool to handle compiling. In Ruby, this same convention came into vogue by way of the Rake tool, a Ruby emulation of the Make tool. Over the years this convention has become increasing popular. The Bundler project brought us the Gemfile. Heroku introduced the Procfile. And now a proverbial explosion of such files seems to be on the doorsteps of Ruby and Rails projects.
The advantage of the Crapfile is that it is visible and even prominent thanks to it’s initial capitalization. In this sense it is the exact opposite of the dot-file. It’s disadvantage is that too many of them tend to clutter up a project directory. This issue could be partially resolved at least if file listings grouped these files together instead of sorting them in with all-caps files, but as of yet no file listing provides for it. Another disadvantage is again the lack of a file extension. As with dot-files, this leaves many tools such as file editors, unable to handle the file optimally. However some tools do allow for an optional extension, e.g. Rakefile.rb is recognized by Rake, so there is some mitigation of the extension issue.
3. Task Directory w/ Special Extension
A third type of common configuration file you will find in Ruby projects is the task/<foo>.<tool> or tasks/<foo>.<tool> file. These files came about as a means of splitting Rakefiles up into topical and shareable units, and indeed this pattern is predominantly used by Rake files, e.g. task/test.rake. But sometimes you will see the pattern used for other tools such as a task/test.watchr file.
The use of task or tasks directory as a conventional location for project configuration files can be advantageous for many project tools. It makes it clear where to find them, the task directory itself is not hidden and does not clutter up a project’s root directory and the design dictates the flexibility of allows multiple, and thus topical, configuration files. However, the issue of the file extension yet again effects these files due to the use the tool name as the extension. Also, there might be a slight downside if a project tool uses the configuration file as a project root marker. This is what Rake does with the Rakefile. While not a show-stopper –after all it can look for task/*.<tool>, it is not quite as ideal for the purpose.
4. Config Directory
Rails came along an introduced a variety of conventions for Ruby-based web applications, but also with ramifications for Ruby project’s in general. One of the conventions introduced is the use of a config directory to house configuration files relevant to the Rails project. Unlike files in the task directory, there is no special extension involved. Generally the whole name of the file is used to identify the purpose of the file. For example config/database.yml is used to configure access to the underlying database.
At first glance, using config seems to have all the advantages of the task directory without the any of downsides associated with using special extensions. Nonetheless, the convention hasn’t caught on for non-Rails projects. The reason is unknown to me, but perhaps it as trivial as the the simple fact that the term config is six-characters long, whereas all conventional project directory names, that have managed to persist, are five or less and most often only three or four characters. It might seem a silly thing to get caught up on, but when a developer is looking at project file listings dozens of times a day or more, the fact the one entry should stick out excessively from the rest is what is commonly called an “eye-sore”. And an “eye-sore” is plenty of reason to deter usage. It is interesting to wonder if Rails had chosen a shorter term, say conf/ or cfg/ if it would have made the jump. In any case, that is mere speculation.
There are a few project’s that support a config directory, and do so to good effect. Cucumber for example will look for either ‘cucumber.yml’, ‘config/cucumber.yml’ or ‘.config/cucumber.yml’.
5. Ordinary File
The last convention really isn’t a convention at all. Some project tools just designate an ordinary file name as the configuration file and expect to find it in the project’s root directory. In many ways, one could argue this is the best choice. It doesn’t suffer from the file extension issue and the files generally sort separate from documentation files which are capitalized. However, it does suffer a bit in that these files don’t stand out in any way as configuration files. And this fact allows them to still make a project seem cluttered if there are more than just a couple of them.
When to Use Which
Hidden dot-files have their place, but they are best reserved for files that the developer will rarely if ever need to edit. Generated files are the best example. The reason for this rule of thumb is simple. If a developer doesn’t see a file, the developer might not realize it already exists or know that he/she may need to edit it. If dot-files do fit your use case, also consider using a dot-directory instead of just a file. This ensures that no additional entries will clutter up the root project directory should more than one configuration file prove necessary or useful.
Crapfiles are not a horrible choice, but as the colorful name suggests, they are not nearly the best choice either. Some developers argue against the use of Crapfiles because the -file suffix is rather redundant. While that’s a fair point, one can also argue that the suffix helps distinguish configuration files from other files. Since Crapfiles tend to standout, they are best suited to configurations that need frequent editing. A word of caution about naming, when you have a tool with a name like “Foreman” and it’s configuration file is called “Procfile”, things are obviously not very intuitive. Consider carefully if perhaps an ordinary file name would suffice instead –it can always be capitalized if it needs to have prominence.
Task files are a good approach for tools that can have multiple configurations or configuration profiles. But even if a project does not support multiple configurations, there is nothing stopping the task directory form holding ordinary files. Which means it could effectively co-opt the advantages of the config directory too.
As for the config directory. If you are developing a Rails project, obviously that is the place for configuration files. While some have argued that config is just for Rails and should not be co-opted for plain Ruby projects, presumably to save confusion, that seems too strict a view. If using the task directory to the same end doesn’t strike your fancy, then config is clearly the best place to put configuration files. The only good exception is when special prominence is needed that only a root level and unhidden file can bring.
Simple uncaptialized ordinary root level files, while seemingly the least problematic, also have nothing that makes them special, which probably accounts for why they are so rarely used. They might as well be placed in the task or config directory to avoid root directory clutter.
Is there a Better Way?
So, is there a better way? That depends on how you look at it. One thing is for certain, there is no perfect way, but that doesn’t exclude the possibility of compelling approach that mitigates most issues.
The overarching problem here is simply the fact that so many conventions exist. With such a variety of naming conventions, it exacerbates the issue of cluttered project directories multiple times over. Another disadvantage for all of these approaches, they make it difficult to share common configuration.
In an effort to create a comprehensive solution, the RC project has been created. RC designate a single file, Config.rb, in which all tool configurations can be defined. A typical entry in this file is for test configuration. For example, to configure the qed test tool we might designate:
profile :coverage do
config :qed do
require 'simplecov'
SimpleCov.start do
coverage_dir 'log/coverage'
end
end
end
RC allows configurations to be grouped into profiles. Tool developers can take advantage of profiles to suite the needs of the tool (or not at all). The major advantage of using a single configuration file is the ability for variant configurations to share settings. Another advantage is to import configurations from other projects. For example, the QED project provides the basic SimpleCov configuration which can be easily reused.
profile :coverage do
config :qed, :from=>[:qed, profile: simplecov]
end
RC is still a work in progress, but the overarching design is in place with numerous advantages over the scattered approaches of the past. Though some of the traditional approaches might still be more suited to very specific cases, most tools can benefit from a unified approach.