Skip to main content

Hydra 1.1

· 4 min read
Omry Yadan

logo

After many months and a lot of hard work by many people, Hydra 1.1 is finally out!
Hydra 1.1 comes with OmegaConf 2.1, which has its own share of awesome new features.

This blog post highlights some of the most prominent features, check the release notes for a complete list of changes:

Major new features in Hydra 1.1​

  • More powerful config composition
    • Every config can now have a Defaults List
    • Composition order of current config can be controlled via the _self_ keyword in the Defaults List
    • Support for composing multiple configs from the same config group
    • Support for configuring the config search path from the primary config
  • Recursive instantiation
  • Experimental callbacks support
  • OmegaConf 2.1:
    • Relative interpolations
    • New OmegaConf interpolation grammar supporting nested interpolations and much more
    • More powerful custom resolvers

More powerful config composition​

Config composition is the key area of improvement in Hydra 1.1.
The biggest change is support for a Defaults List in any config, and not just the primary config.

This enables many new capabilities:

  • Any config can now "extend" other configs. This enables config files to be associated with a Structured Config schema and to extend other config files
  • A top level "experiment config" can now override the Defaults List as well as config values
  • Complex frameworks can now have their own Defaults List, reducing boilerplate

Other related changes include the ability to change the order a config is composed relative to config in its Defaults List by adding _self_ to the Defaults List and the ability to use multiple configs from the same config group.

Learn more:

Object instantiation enhancements​

hydra.utils.instantiate() now instantiates nested objects recursively.

Other enhancements include:

  • Support for positional arguments via the _args_ config key
  • Support for parameter conversion strategy was added via the _convert_ config key

Learn more here.

Hydra callbacks​

A new experimental mechanism for user defined callbacks was added. Callbacks enable user code to be executed automatically at various points in the lifecycle of your application. There are many potential use cases for this, for example automatic registration with your favorite experiment-tracking service.

Learn more here.

OmegaConf 2.1 highlights​

OmegaConf 2.1 includes many enhancements, bug fixes, and performance improvements.

Relative interpolations​

Relative interpolations enable accessing a config node relative to the node defining the interpolation:

x: 10
b:
  y: 20
  a: {x}    # 10, absolute interpolation
  b: ${.y}  # 20, relative interpolation
  c: ${..x} # 10, relative interpolation

Nested interpolations​

OmegaConf 2.1 adds a new interpolation grammar supporting more sophisticated usage of interpolations.
In the following example, the default value to use if the environment variable DB_USER does not exist is defined in the default_user config node: 

default_user: root
db_user: ${oc.env:DB_USER,${default_user}}

More powerful custom resolvers​

OmegaConf custom resolvers can now access parent config node or the config root by defining keyword parameters named _parent_ and _root_.

In the example below, we use _parent_ to implement a sum function that defaults to 0 if the node does not exist:

def sum(a, b, *, _parent_):
  return _parent_.get(a, 0) + _parent_.get(b, 0)

OmegaConf.register_new_resolver("sum", sum)
cfg = OmegaConf.create({
  "node": {
    "a": 1,
    "b": 2,
    "a_plus_b": "${sum:a,b}",
    "a_plus_z": "${sum:a,z}",
  },
})
print(cfg.node.a_plus_b)  # 3
print(cfg.node.a_plus_z)  # 1

Other notable improvements​

  • Config composition, especially for large configs - is significantly faster.
  • OmegaConf.resolve(cfg) can be used for in-place interpolation resolution on a config object
  • Improved compatibility of OmegaConf config objects with plain dict and list
  • Support for bracketed style (foo.bar is equivalent to foo[bar]), this covers interpolations and OmegaConf.{update, select} usage
  • PyDev.Debugger integration for easier debugging of config objects in PyCharm and VSCode

Migrating from 1.0​

Hydra 1.1 is a major release. For most people, migrating from 1.0 to 1.1 will be smooth. However, there are some breaking changes listed in the release notes of OmegaConf 2.1 and Hydra 1.1. Most changes come with a deprecation warning pointing to a specific migration guide page. Please feel free to reach out for help if you see a change in behavior that is not mentioned in the release notes.

That's it for now, take Hydra 1.1 for a spin!