Skip to main content
Version: 1.1

Basic Override syntax#

You can manipulate your configuration with overrides (via the command line or the Compose API). This includes:

  • Modifying the the Defaults List
  • Modifying the config object

Overrides matching a config group are modifying the Defaults List; The rest are manipulating the config object.

Basic examples#

Modifying the Config Object#

  • Overriding a config value : foo.bar=value
  • Appending a config value : +foo.bar=value
  • Appending or overriding a config value : ++foo.bar=value
  • Removing a config value : ~foo.bar, ~foo.bar=value

Modifying the Defaults List#

  • Overriding selected Option: db=mysql
  • Appending to Defaults List: +db=mysql
  • Deleting from Defaults List: ~db, ~db=mysql

Grammar#

Hydra supports a rich DSL in the command line. Below are the parser rules from grammar. You can see the full Lexer and Parser definitions on GitHub.

OverrideParser.g4
// High-level command-line override.
override: (
key EQUAL value? // key=value, key= (for empty value)
| TILDE key (EQUAL value?)? // ~key | ~key=value
| PLUS PLUS? key EQUAL value? // +key= | +key=value | ++key=value
) EOF;
// Keys.
key : packageOrGroup (AT package)?; // key | [email protected]
packageOrGroup: package | ID (SLASH ID)+; // db, hydra/launcher
package: ( | ID | KEY_SPECIAL | DOT_PATH); // db, $db, hydra.launcher, or the empty
// Elements (that may be swept over).
value: element | simpleChoiceSweep;
element:
primitive
| listContainer
| dictContainer
| function
;
simpleChoiceSweep:
element (COMMA element)+ // value1,value2,value3
;
// Functions.
argName: ID EQUAL;
function: ID POPEN (argName? element (COMMA argName? element )* )? PCLOSE;
// Data structures.
listContainer: BRACKET_OPEN // [], [1,2,3], [a,b,[1,2]]
(element(COMMA element)*)?
BRACKET_CLOSE;
dictContainer: BRACE_OPEN (dictKeyValuePair (COMMA dictKeyValuePair)*)? BRACE_CLOSE; // {}, {a:10,b:20}
dictKeyValuePair: dictKey COLON element;
// Primitive types.
primitive:
QUOTED_VALUE // 'hello world', "hello world"
| ( ID // foo_10
| NULL // null, NULL
| INT // 0, 10, -20, 1_000_000
| FLOAT // 3.14, -20.0, 1e-1, -10e3
| BOOL // true, TrUe, false, False
| INTERPOLATION // ${foo.bar}, ${oc.env:USER,me}
| UNQUOTED_CHAR // /, -, \, +, ., $, %, *, @, ?
| COLON // :
| ESC // \\, \(, \), \[, \], \{, \}, \:, \=, \ , \\t, \,
| WS // whitespaces
)+;
// Same as `primitive` except that `COLON` and `INTERPOLATION` are not allowed.
dictKey:
QUOTED_VALUE // 'hello world', "hello world"
| ( ID // foo_10
| NULL // null, NULL
| INT // 0, 10, -20, 1_000_000
| FLOAT // 3.14, -20.0, 1e-1, -10e3
| BOOL // true, TrUe, false, False
| UNQUOTED_CHAR // /, -, \, +, ., $, %, *, @, ?
| ESC // \\, \(, \), \[, \], \{, \}, \:, \=, \ , \\t, \,
| WS // whitespaces
)+;

Elements#

Key#

Key is the component before the =. A few examples:

foo.bar # A config key
hydra/launcher # A config group
[email protected] # A config group assigned to the package pkg
[email protected]:pkg2 # A config group changing the package from pkg1 to pkg2

Quoted values#

Hydra supports both double quotes and single quoted values. Quoted strings can accept any value between the quotes, but some characters need escaping:

  • to include a single quote in a single quoted string, use \' (for double quotes in a double quoted string, use \")
  • any sequence of \ characters preceding a quote (either an escaped quote as described in the previous point, or the closing quote) must be escaped by doubling the number of \
Double quotes
"hello there"
"escaped \"double quote\""
"the path is C:\\\"some folder\"\\"
"1,2,3"
"{a:10} ${xyz}"
"'single quoted string'"
Single quotes
'hello there'
'escaped \'single quote\''
'the path is C:\\\'some folder\'\\'
'1,2,3'
'{a:10} ${xyz}'
'"double quoted string"'

Whitespaces in unquoted values#

Unquoted Override values can contain non leading or trailing whitespaces. For example, msg=hello world is a legal override (key is msg and value is the string hello world). Normally, your shell will interpret values with whitespaces as being multiple parameters (key=a b would be interpreted as key=a and b). To prevent this you can quote them with a single quote. For example:

$ python my_app.py 'msg=hello world'

Note that trailing and leading whitespace are ignored, the above is equivalent to:

$ python my_app.py 'msg= hello world '

Escaped characters in unquoted values#

Some otherwise special characters may be included in unquoted values by escaping them with a \. These characters are: \()[]{}:=, \t (the last two ones being the whitespace and tab characters).

As an example, in the following dir is set to the string job{a=1,b=2,c=3}:

$ python my_app.py 'dir=job\{a\=1\,b\=2\,c\=3\}'

Primitives#

  • id : oompa10, loompa_12
  • null: null
  • int: 10, -20, 0, 1_000_000.
  • float: 3.14, -10e6, inf, -inf, nan.
  • bool: true, false
  • dot_path: foo.bar
  • interpolation: ${foo.bar}, ${oc.env:USER,me}

Constants (null, true, false, inf, nan) are case-insensitive.

important

Always single-quote interpolations in the shell, to prevent replacement with shell variables:

$ python my_app.py 'dir=/root/${name}'

In addition, more complex interpolations containing special characters may require being passed within a quoted value (note the extra double quotes surrounding the interpolation):

$ python my_app.py 'dir="${get_dir: {root: /root, name: ${name}}}"'

Dictionaries and Lists#

Lists#

foo=[1,2,3]
nested=[a,[b,[c]]]

Dictionaries#

foo={a:10,b:20}
nested={a:10,b:{c:30,d:40}}

Dictionaries are merged, not assigned. The following example illustrates the point:

Input config
db:
driver: mysql
user: ???
pass: ???
db={user:root,pass:1234}
db:
driver: mysql
user: root
pass: 1234
important

Always single-quote overrides that contains dicts and lists in the shell.

Sweeper syntax#

A choice sweep is comma separated list with two or more elements:

key=a,b # Simple sweep: ChoiceSweep(a, b)
key="a,b","c,d" # Elements can be quoted strings, ChoiceSweep("a,b", "c,d")
key=[a,b],[c,d] # Elements can be real lists, ChoiceSweep([a,b], [c,d])
key={a:10, b:20},{c:30,d:40} # And dictionaries: ChoiceSweep({a:10, b:20}, {c:30,d:40})

More sweeping options are described in the Extended Grammar page.

important

You may need to quote your choice sweep in the shell.

Functions#

Hydra supports several functions in the command line. See the Extended Grammar page for more information.

Working with your shell#

All shells interprets command line inputs and may change what is passed to the process. A good way to determine what the shell is doing to your command is to echo it.

# bash output
$ echo foo_{a:10,b:20} ${HOME} [b,c]*
foo_a:10 foo_b:20 /home/omry build_helpers
$ echo 'foo_{a:10,b:20}' '${HOME}' '[b,c]*'
foo_{a:10,b:20} ${HOME} [b,c]*

If in doubt, quote a command line element with a single quote (').

If you want to pass quotes to Hydra in a shell quoted string, it's best to pass double quotes.

$ echo '"hello world"'
"hello world"

You can use some shell specific commands to change their behavior, but the cost will be that their behavior will change.

Bash#

You can disable braces expansion, filename generation (globing) and hist expansion. Please note that this will change your shell behavior for the current session.

$ set +o braceexpand -o noglob +o histexpand
$ echo key1={a:10,b:20} key2=${HOME} key=[b]*
key1={a:10,b:20} key2=/home/omry key=[b]*
# does not help with () though:
$ echo key=choice(a,b,c)
bash: syntax error near unexpected token '('
$ echo 'key=choice(a,b,c)'
key=choice(a,b,c)

Other shells#

Send a PR to add information about your favorite shell here.

Last updated on by Jasha10