Skip to main content

Hydra 1.2

· 4 min read
Padraig Brady

logo

After many months and a lot of hard work by many people, Hydra 1.2 is released! Hydra 1.2 comes with OmegaConf 2.2, which has its own share of improvements.

This blog post highlights some of the most prominent features, check the release notes for a complete list of changes:

Major new features in Hydra 1.2​

  • Easier integration with existing systems
    • Support not changing working directory at runtime
    • Default to not implicitly adding directories to the config path
  • Improved support for reproducible experiments
    • Support defining multirun mode and sweeping parameters through config
    • Improved callback support for logging / persisting job runs
    • A new --experimental-rerun option to reproduce persisted single runs
  • Improved instantiate API functionality
    • Support for instances partially defined from config, via a _partial_ keyword
    • Accept ListConfig/list-type config as top-level input
  • Better alignment with ecosystem versions
    • Support for Python 3.10, and ANTLR 4.9
  • OmegaConf 2.2:
    • More flexible type hints in structured configs
    • Native support for bytes and pathlib.Path

Object instantiation enhancements​

  • Lists can now be passed directly to the instantiate API. For example one can now do:
from hydra.utils import instantiate

lst = [
    {"_target_": "pathlib.Path", "_args_": ["foo"]},
    {"_target_": "pathlib.Path", "_args_": ["bar"]},
]

paths = instantiate(lst)
print(paths)

Resulting in:

$ python demo.py
[PosixPath('foo'), PosixPath('bar')]

OmegaConf 2.2 highlights​

More flexible type hints in structured configs​

OmegaConf 2.2's structured configs support runtime type checking for an expanded set of type hints. It is now possible to use nested container types (e.g. dict-of-dict or list-of-list), unions of primitive types, and containers with optional element types.

Here is an example demonstrating these new capabilities:

from dataclasses import dataclass
from typing import Dict, List, Optional, Union
from omegaconf import OmegaConf

@dataclass
class DemoConfig:
    union: Union[int, str, bool]
    dict_of_union: Dict[str, Union[int, str]]
    list_of_dict: List[Dict[int, float]]
    dict_of_optional: Dict[str, Optional[int]]

cfg = OmegaConf.structured({"foo": DemoConfig})
cfg.foo.dict_of_union = {"abc": 123}  # ok
cfg.foo.dict_of_union = {"abc": 10.1}  # raises ValidationError!
# Value '10.1' of type 'float' is incompatible with type hint 'Union[int, str]'

Native support for bytes and pathlib.Path​

OmegaConf now supports binary data via Python's bytes type.

cfg = OmegaConf.create({b"binary_key": b"binary_value"})

In addition, OmegaConf now supports pathlib.Path instances as config values, easing workflows that involve the file system.

from pathlib import Path
cfg.my_homedir = Path.home()
assert cfg.my_homedir.is_dir()

The bytes and pathlib.Path types can be used as type hints in structured config class definitions, and configs containing binary and Path data can be round-tripped to/from yaml files via OmegaConf's save/load/to_yaml/create methods.

yaml_data = OmegaConf.to_yaml(cfg)
cfg2 = OmegaConf.create(yaml_data)
assert cfg2[b"binary_key"] == b"binary_value"
assert isinstance(cfg2.my_homedir, Path)

Migrating from 1.1​

Hydra 1.2 is a major release. For most people, migrating from 1.1 to 1.2 will be smooth. In addition, for this release we introduce support for more compatible upgrades through the version_base mechanism. New users are encouraged to use the latest defaults by setting version_base=None with @hydra.main() and hydra.initialize(), while existing users have more control over what potentially incompatible changes are introduced when upgrading to Hydra 1.2. Please see the "Behavior changes" section of the Hydra 1.2 release notes for details. OmegaConf 2.2 also has some API changes and deprecations (not protected by version_base), detailed in its release notes. Please feel free to reach out for help if you see a change in behavior that is not mentioned in the release notes.

That's it for now, take Hydra 1.2 for a spin!