Customizing working directory pattern

 

Hydra automatically creates an output directory used to store log files and save yaml configs. This directory can be configured by setting hydra.run.dir (for single hydra runs) or hydra.sweep.dir/hydra.sweep.subdir (for multirun sweeps). At runtime, the path of the output directory can be accessed via the hydra.runtime.output_dir variable. Below are a few examples of customizing output directory patterns.

Configuration for run

Run output directory grouped by date:

hydra:
  run:
    dir: ./outputs/${now:%Y-%m-%d}/${now:%H-%M-%S}

Run output directory grouped by job name:

hydra:
  run:
    dir: outputs/${hydra.job.name}/${now:%Y-%m-%d_%H-%M-%S}

Run output directory can contain user configuration variables:

hydra:
  run:
    dir: outputs/${now:%Y-%m-%d_%H-%M-%S}/opt:${optimizer.type}

Configuration for multirun

We will run the application with same command but different configurations:

python my_app.py --multirun a=a1,a2,a3 

Default multirun dir configurations:

config.yaml

hydra:
  sweep:
    dir: multirun/${now:%Y-%m-%d}/${now:%H-%M-%S}
    subdir: ${hydra.job.num}

workding dir created

$ tree my_app -d
my_app
├── 0
├── 1
└── 2

Similar configuration patterns in run can be applied to config multirun dir as well.

For example, multirun output directory grouped by job name, and sub dir by job num:

config.yaml

hydra:
  sweep:
    dir: ${hydra.job.name}
    subdir: ${hydra.job.num}

workding dir created

$ tree my_app -d
my_app
├── 0
├── 1
└── 2

Using hydra.job.override_dirname

 

This field is populated automatically using your command line arguments and is typically being used as a part of your output directory pattern. It is meant to be used along with the configuration for working dir, especially in hydra.sweep.subdir.

If we run the example application with the following commandline overrides and configs:

python my_app.py --multirun batch_size=32 learning_rate=0.1,0.01

config.yaml

hydra:
  sweep:
    dir: multirun
    subdir: ${hydra.job.override_dirname}

working dir created

$ tree multirun -d
multirun
├── batch_size=32,learning_rate=0.01
└── batch_size=32,learning_rate=0.1

You can further customized the output dir creation by configuringhydra.job.override_dirname.

In particular, the separator char = and the item separator char , can be modified by overriding hydra.job.config.override_dirname.kv_sep and hydra.job.config.override_dirname.item_sep. Command line override keys can also be automatically excluded from the generated override_dirname.

An example of a case where the exclude is useful is a random seed.

hydra:
  run:
    dir: output/${hydra.job.override_dirname}/seed=${seed}
  job:
    config:
      override_dirname:
        exclude_keys:
          - seed

With this configuration, running

$ python my_app.py --multirun  batch_size=32 learning_rate=0.1,0.01 seed=1,2

Would result in a directory structure like:

$ tree multirun -d
multirun
├── batch_size=32,learning_rate=0.01
│   ├── seed=1
│   └── seed=2
└── batch_size=32,learning_rate=0.1
    ├── seed=1
    └── seed=2