Version: 0.11

Config composition

As software gets more complex, we resort to modularity and composition to keep it manageable. We can do the same with configs: suppose we want our working example to support multiple databases, with multiple schemas per database, and different UIs. We wouldn't write a separate class for each permutation of db, schema and UI, so we shouldn't write separate configs either. We use the same solution in configuration as in writing the underlying software: composition.

To do this in Hydra, we first add a schema and a ui config group:

โ”œโ”€โ”€ conf
โ”‚ย ย  โ”œโ”€โ”€ config.yaml
โ”‚ย ย  โ”œโ”€โ”€ db
โ”‚ย ย  โ”‚ย ย  โ”œโ”€โ”€ mysql.yaml
โ”‚ย ย  โ”‚ย ย  โ””โ”€โ”€ postgresql.yaml
โ”‚ย ย  โ”œโ”€โ”€ schema
โ”‚ย ย  โ”‚ย ย  โ”œโ”€โ”€ school.yaml
โ”‚ย ย  โ”‚ย ย  โ”œโ”€โ”€ support.yaml
โ”‚ย ย  โ”‚ย ย  โ””โ”€โ”€ warehouse.yaml
โ”‚ย ย  โ””โ”€โ”€ ui
โ”‚ย ย  โ”œโ”€โ”€ full.yaml
โ”‚ย ย  โ””โ”€โ”€ view.yaml
โ””โ”€โ”€ my_app.py

With these configs, we already have 12 possible combinations. Without composition we would need 12 separate configs, and a single change (such as renaming db.user to db.username) would need to be done separately in every one of them.

This is a maintainability nightmare -- but composition can come to the rescue.

Configuration file: config.yaml

defaults:
- db: mysql
- ui: full
- schema: school

The defaults are ordered:

  • If there are two configurations that defines the same value, the second one would win.
  • If two configurations are contributing to the same dictionary the result would be the combined dictionary.

When running this, we will compose a configuration with mysql, full ui and the school database schema (which we are seeing for the first time here):

$ python my_app.py
db:
driver: mysql
pass: secret
user: omry
schema:
database: school
tables:
- fields:
- name: string
- class: int
name: students
- fields:
- profession: string
- time: data
- class: int
name: exams
ui:
windows:
create_db: true
view: true

In much the same way you can compose any of the other 11 configurations by adding appropriate overrides such as db=postgresql.

Last updated on by Omry Yadan