Skip to main content
Version: 1.1

Structured Config schema

We have seen how to use Structured Configs as configuration, but they can also be used as a schema (i.e. validating configuration files). To achieve this, we will follow the common pattern of Extending Configs - but instead of extending another config file, we will extend a Structured Config.

This page shows how to validate the config files config.yaml, db/mysql.yaml and db/postgresql.yaml against a Structured Config schema.

Validating against a schema in the same config group#

ย Example

Given the config directory structure:

conf/
โ”œโ”€โ”€ config.yaml
โ””โ”€โ”€ db
โ”œโ”€โ”€ mysql.yaml
โ””โ”€โ”€ postgresql.yaml

We will add Structured Config schema for each of the config files above and store in the Config Store as base_config, db/base_mysql and db/base_postgresql.

Then, we will use the Defaults List in each config to specify its base config as follows:

config.yaml
defaults:
- base_config
- db: mysql
debug: true
db/mysql.yaml
defaults:
- base_mysql
user: omry
password: secret
db/postgresql.yaml
defaults:
- base_postgresql
user: postgres_user
password: drowssap

One difference in the source code is that we have removed the Defaults List from the Config dataclass. The primary Defaults List will come from config.yaml.

my_app.py (Click to expand)
@dataclass
class DBConfig:
driver: str = MISSING
host: str = "localhost"
port: int = MISSING
@dataclass
class MySQLConfig(DBConfig):
driver: str = "mysql"
port: int = 3306
user: str = MISSING
password: str = MISSING
@dataclass
class PostGreSQLConfig(DBConfig):
driver: str = "postgresql"
user: str = MISSING
port: int = 5432
password: str = MISSING
timeout: int = 10
@dataclass
class Config:
db: DBConfig = MISSING
debug: bool = False
cs = ConfigStore.instance()
cs.store(name="base_config", node=Config)
cs.store(group="db", name="base_mysql", node=MySQLConfig)
cs.store(group="db", name="base_postgresql", node=PostGreSQLConfig)
@hydra.main(config_path="conf", config_name="config")
def my_app(cfg: Config) -> None:
print(OmegaConf.to_yaml(cfg))
if __name__ == "__main__":
my_app()

When Hydra composes the final config object it will use the config schemas as specified in the Default Lists.
Like before, Hydra will catch user errors in the command line:

$ python my_app.py db.port=fail
Error merging override db.port=fail
Value 'fail' could not be converted to Integer
full_key: db.port
object_type=MySQLConfig
Use --info commands to see how a config was composed (Expand)
$ python my_app.py --info defaults-tree
Defaults Tree
*************
<root>:
hydra/config:
hydra/output: default
hydra/launcher: basic
hydra/sweeper: basic
hydra/help: default
hydra/hydra_help: default
hydra/hydra_logging: default
hydra/job_logging: default
_self_
config:
base_config
db: mysql:
db/base_mysql
_self_
_self_
$ python my_app.py --info defaults
Defaults List
*************
| Config path | Package | _self_ | Parent |
------------------------------------------------------------------------------
| hydra/output/default | hydra | False | hydra/config |
| hydra/launcher/basic | hydra.launcher | False | hydra/config |
| hydra/sweeper/basic | hydra.sweeper | False | hydra/config |
| hydra/help/default | hydra.help | False | hydra/config |
| hydra/hydra_help/default | hydra.hydra_help | False | hydra/config |
| hydra/hydra_logging/default | hydra.hydra_logging | False | hydra/config |
| hydra/job_logging/default | hydra.job_logging | False | hydra/config |
| hydra/config | hydra | True | <root> |
| base_config | | False | config |
| db/base_mysql | db | False | db/mysql |
| db/mysql | db | True | config |
| config | | True | <root> |
------------------------------------------------------------------------------

Validating against a schema from a different config group#

ย Example

In the above example, the schema we used was stored in the same config group. This is not always the case, for example - A library might provide schemas in its own config group.

Here is a modified version of the example above, where a mock database_lib is providing the schemas we want to validate against.

my_app.py
import database_lib
@dataclass
class Config:
db: database_lib.DBConfig = MISSING
debug: bool = False
cs = ConfigStore.instance()
cs.store(name="base_config", node=Config)
# database_lib registers its configs
# in database_lib/db
database_lib.register_configs()
@hydra.main(
config_path="conf",
config_name="config",
)
def my_app(cfg: Config) -> None:
print(OmegaConf.to_yaml(cfg))
if __name__ == "__main__":
my_app()
database_lib.py
@dataclass
class DBConfig:
...
@dataclass
class MySQLConfig(DBConfig):
...
@dataclass
class PostGreSQLConfig(DBConfig):
...
def register_configs() -> None:
cs = ConfigStore.instance()
cs.store(
group="database_lib/db",
name="mysql",
node=MySQLConfig,
)
cs.store(
group="database_lib/db",
name="postgresql",
node=PostGreSQLConfig,
)

The Defaults List entry for the base config is slightly different:

db/mysql.yaml
defaults:
- /database_lib/db/[email protected]_here_
user: omry
password: secret
db/postgresql.yaml
defaults:
- /database_lib/db/[email protected]_here_
user: postgres_user
password: drowssap
  • We refer to the config with an absolute path because it is outside the subtree of the db config group.
  • we override the package to _here_ to ensure that the package of the schema is the same as the package of the config it's validating.
Last updated on by Jasha10