Skip to main content
Version: 1.0

Dynamic schema with many configs

ย Example

In this page we will see how to get runtime type safety for configs with dynamic schema. Our top level config contains a single field - db, with the type DBConfig. Based on user choice, we would like its type to be either MySQLConfig or PostGreSQLConfig at runtime. The two schemas differs: config files that are appropriate for one are inappropriate for the other.

For each of the two schemas, we have two options - one for prod and one for staging:

Config directory
โ”œโ”€โ”€ config.yaml
โ””โ”€โ”€ db
โ”œโ”€โ”€ mysql_prod.yaml
โ”œโ”€โ”€ mysql_staging.yaml
โ”œโ”€โ”€ postgresql_prod.yaml
โ””โ”€โ”€ postgresql_staging.yaml
class DBConfig:
driver: str = MISSING
host: str = MISSING
class MySQLConfig(DBConfig):
driver: str = "mysql"
encoding: str = MISSING
class PostGreSQLConfig(DBConfig):
driver: str = "postgresql"
timeout: int = MISSING
class Config:
db: DBConfig = MISSING
cs = ConfigStore.instance()"schema/db", name="mysql", node=MySQLConfig, package="db")"schema/db", name="postgresql", node=PostGreSQLConfig, package="db")"config", node=Config)
@hydra.main(config_path="conf", config_name="config")
def my_app(cfg: Config) -> None:
if __name__ == "__main__":

When composing the config, we need to select both the schema and the actual config group option. This is what the defaults list looks like:

- schema/db: mysql
- db: mysql_staging

Let's dissect the how we store the schemas into the ConfigStore:"schema/db", name="mysql", node=MySQLConfig, package="db")

There are several notable things here:

  • We use the group schema/db and not db.
    Config Groups are mutually exclusive, only one option can be selected from a Config Group. We want to select both the schema and the config. Storing all schemas in subgroups of the config group schema is good practice. This also helps in preventing name collisions.
  • We need to specify the package to be db. By default, the package for configs stored in the ConfigStore is _group_. We want to schematize db and not schema.db in the config so we have to override that.

By default, we get the mysql staging config:

$ python
driver: mysql
host: mysql001.staging
encoding: utf-8

We can change both the schema and the config:

$ python schema/db=postgresql db=postgresql_prod
driver: postgresql
timeout: 10

If we try to use a postgresql config without changing the schema as well we will get an error:

$ python db=postgresql_prod
Error merging db=postgresql_prod
Key 'timeout' not in 'MySQLConfig'
full_key: db.timeout
Last updated on by Jasha10