Skip to main content
Version: Next

Basic Override syntax#

You can manipulate your configuration with overrides (via the command line or the Compose API). This includes:

  • Modifying the Defaults List
  • Modifying the config object

Overrides matching a config group are modifying the Defaults List; The rest are manipulating the config object.

Basic examples#

Modifying the Config Object#

  • Overriding a config value : foo.bar=value
  • Appending a config value : +foo.bar=value
  • Appending or overriding a config value : ++foo.bar=value
  • Removing a config value : ~foo.bar, ~foo.bar=value

Modifying the Defaults List#

  • Overriding selected Option: db=mysql
  • Appending to Defaults List: +db=mysql
  • Deleting from Defaults List: ~db, ~db=mysql

Grammar#

Hydra supports a rich DSL in the command line. Below are the parser rules from grammar. You can see the full Lexer and Parser definitions on GitHub.

OverrideParser.g4
// High-level command-line override.
override: (      key EQUAL value?                           // key=value, key= (for empty value)    | TILDE key (EQUAL value?)?                  // ~key | ~key=value    | PLUS PLUS? key EQUAL value?                // +key= | +key=value | ++key=value) EOF;
// Key:key : packageOrGroup (AT package)?;              // key | [email protected]
packageOrGroup: package | ID (SLASH ID)+;        // db, hydra/launcherpackage: ( | ID | KEY_SPECIAL | DOT_PATH);       // db, $db, hydra.launcher, or the empty (for _global_ package)
// Elements (that may be swept over).
value: element | simpleChoiceSweep;
element:      primitive    | listContainer    | dictContainer    | function;
simpleChoiceSweep:      element (COMMA element)+                   // value1,value2,value3;
// Functions.
argName: ID EQUAL;function: ID POPEN (argName? element (COMMA argName? element )* )? PCLOSE;
// Data structures.
listContainer: BRACKET_OPEN                      // [], [1,2,3], [a,b,[1,2]]    (element(COMMA element)*)?BRACKET_CLOSE;
dictContainer: BRACE_OPEN (dictKeyValuePair (COMMA dictKeyValuePair)*)? BRACE_CLOSE;  // {}, {a:10,b:20}dictKeyValuePair: dictKey COLON element;
// Primitive types.
primitive:      QUOTED_VALUE                               // 'hello world', "hello world"    | (   ID                                     // foo_10        | NULL                                   // null, NULL        | INT                                    // 0, 10, -20, 1_000_000        | FLOAT                                  // 3.14, -20.0, 1e-1, -10e3        | BOOL                                   // true, TrUe, false, False        | INTERPOLATION                          // ${foo.bar}, ${oc.env:USER,me}        | UNQUOTED_CHAR                          // /, -, \, +, ., $, %, *, @, ?        | COLON                                  // :        | ESC                                    // \\, \(, \), \[, \], \{, \}, \:, \=, \ , \\t, \,        | WS                                     // whitespaces    )+;
// Same as `primitive` except that `COLON` and `INTERPOLATION` are not allowed.dictKey:    (   ID                                     // foo_10      | NULL                                   // null, NULL      | INT                                    // 0, 10, -20, 1_000_000      | FLOAT                                  // 3.14, -20.0, 1e-1, -10e3      | BOOL                                   // true, TrUe, false, False      | UNQUOTED_CHAR                          // /, -, \, +, ., $, %, *, @, ?      | ESC                                    // \\, \(, \), \[, \], \{, \}, \:, \=, \ , \\t, \,      | WS                                     // whitespaces    )+;

Elements#

Key#

Key is the component before the =. A few examples:

foo.bar           # A config keyhydra/launcher    # A config group[email protected]         # A config group assigned to the package pkg[email protected]:pkg2   # A config group changing the package from pkg1 to pkg2

Quoted values#

Hydra supports both double quotes and single quoted values. Quoted strings can accept any value between the quotes, but some characters need escaping:

  • to include a single quote in a single quoted string, use \' (for double quotes in a double quoted string, use \")
  • any sequence of \ characters preceding a quote (either an escaped quote as described in the previous point, or the closing quote) must be escaped by doubling the number of \
Double quotes
"hello there""escaped \"double quote\"""the path is C:\\\"some folder\"\\""1,2,3""{a:10} ${xyz}""'single quoted string'"
Single quotes
'hello there''escaped \'single quote\'''the path is C:\\\'some folder\'\\''1,2,3''{a:10} ${xyz}''"double quoted string"'

Whitespaces in unquoted values#

Unquoted Override values can contain non leading or trailing whitespaces. For example, msg=hello world is a legal override (key is msg and value is the string hello world). Normally, your shell will interpret values with whitespaces as being multiple parameters (key=a b would be interpreted as key=a and b). To prevent this you can quote them with a single quote. For example:

$ python my_app.py 'msg=hello world'

Note that trailing and leading whitespace are ignored, the above is equivalent to:

$ python my_app.py 'msg=    hello world    '

Escaped characters in unquoted values#

Some otherwise special characters may be included in unquoted values by escaping them with a \. These characters are: \()[]{}:=, \t (the last two ones being the whitespace and tab characters).

As an example, in the following dir is set to the string job{a=1,b=2,c=3}:

$ python my_app.py 'dir=job\{a\=1\,b\=2\,c\=3\}'

Primitives#

  • id : oompa10, loompa_12
  • null: null
  • int: 10, -20, 0, 1_000_000.
  • float: 3.14, -10e6, inf, -inf, nan.
  • bool: true, false
  • dot_path: foo.bar
  • interpolation: ${foo.bar}, ${oc.env:USER,me}

Constants (null, true, false, inf, nan) are case-insensitive.

important

Always single-quote interpolations in the shell, to prevent replacement with shell variables:

$ python my_app.py 'dir=/root/${name}'

In addition, more complex interpolations containing special characters may require being passed within a quoted value (note the extra double quotes surrounding the interpolation):

$ python my_app.py 'dir="${get_dir: {root: /root, name: ${name}}}"'

Dictionaries and Lists#

Lists#

foo=[1,2,3]nested=[a,[b,[c]]]

Dictionaries#

foo={a:10,b:20}nested={a:10,b:{c:30,d:40}}

Dictionaries are merged, not assigned. The following example illustrates the point:

Input config
db:  driver: mysql  user: ???  pass: ???
db={user:root,pass:1234}
db:  driver: mysql  user: root  pass: 1234
important

Always single-quote overrides that contains dicts and lists in the shell.

Sweeper syntax#

A choice sweep is comma separated list with two or more elements:

key=a,b                       # Simple sweep: ChoiceSweep(a, b)key="a,b","c,d"               # Elements can be quoted strings, ChoiceSweep("a,b", "c,d")key=[a,b],[c,d]               # Elements can be real lists, ChoiceSweep([a,b], [c,d])key={a:10, b:20},{c:30,d:40}  # And dictionaries: ChoiceSweep({a:10, b:20}, {c:30,d:40})

More sweeping options are described in the Extended Grammar page.

important

You may need to quote your choice sweep in the shell.

Functions#

Hydra supports several functions in the command line. See the Extended Grammar page for more information.

Working with your shell#

All shells interprets command line inputs and may change what is passed to the process. A good way to determine what the shell is doing to your command is to echo it.

# bash output$ echo foo_{a:10,b:20} ${HOME} [b,c]*foo_a:10 foo_b:20 /home/omry build_helpers$ echo 'foo_{a:10,b:20}' '${HOME}' '[b,c]*'foo_{a:10,b:20} ${HOME} [b,c]*

If in doubt, quote a command line element with a single quote (').

If you want to pass quotes to Hydra in a shell quoted string, it's best to pass double quotes.

$ echo '"hello world"'"hello world"

You can use some shell specific commands to change their behavior, but the cost will be that their behavior will change.

Bash#

You can disable braces expansion, filename generation (globing) and hist expansion. Please note that this will change your shell behavior for the current session.

$ set +o braceexpand -o noglob +o histexpand$ echo key1={a:10,b:20} key2=${HOME} key=[b]*key1={a:10,b:20} key2=/home/omry key=[b]*# does not help with () though:$ echo key=choice(a,b,c)bash: syntax error near unexpected token '('$ echo 'key=choice(a,b,c)'key=choice(a,b,c)

Other shells#

Send a PR to add information about your favorite shell here.