Configuring Experiments
#
ProblemA common problem is maintaining multiple configurations of an application. This can get especially tedious when the configuration differences span multiple dimensions. This pattern shows how to cleanly support multiple configurations, with each configuration file only specifying the changes to the master (default) configuration.
#
SolutionCreate a config file specifying the overrides to the default configuration, and then call it via the command line.
e.g. $ python my_app.py +experiment=fast_mode
.
To avoid clutter, we place the experiment config files in dedicated config group called experiment.
#
ExampleIn this example, we will create configurations for each of the server and database pairings that we want to benchmark.
The default configuration is:
The benchmark config files specify the deltas from the default configuration:
Key concepts:
- # @package _global_
Changes specified in this config should be interpreted as relative to the _global_ package.
We could instead place nglite.yaml and aplite.yaml next to config.yaml and omit this line. - The overrides of /db and /server are absolute paths.
This is necessary because they are outside of the experiment directory.
Running the experiments from the command line requires prefixing the experiment choice with a +
.
The experiment config group is an addition, not an override.
#
Sweeping over experimentsThis approach also enables sweeping over those experiments to easily compare their results:
To run all the experiment, use the glob syntax: