Skip to main content
Version: 1.0

Basic Override syntax​

You can manipulate your configuration with overrides (via the command line or the Compose API). This includes:

  • Modifying the Defaults List
  • Modifying the config object

Overrides matching a config group are modifying the Defaults List; The rest are manipulating the config object.

Basic examples​

Modifying the Config Object​

  • Overriding a config value : foo.bar=value
  • Appending a config value : +foo.bar=value
  • Removing a config value : ~foo.bar, ~foo.bar=value

Modifying the Defaults List​

  • Overriding selected Option: db=mysql
  • Appending to defaults: +db=mysql
  • Deleting from defaults: ~db, ~db=mysql

Grammar​

Hydra supports a rich DSL in the command line. Below are the parser rules from grammar. You can see the full grammar on GitHub (lexer and parser).

OverrideParser.g4
// High-level command-line override.

override: (
key EQUAL value? // key=value, key= (for empty value)
| TILDE key (EQUAL value?)? // ~key | ~key=value
| PLUS key EQUAL value? // +key= | +key=value
) EOF;

// Keys.

key : packageOrGroup (AT package)?; // key | group@pkg

packageOrGroup: package | ID (SLASH ID)+; // db, hydra/launcher
package: (ID | DOT_PATH); // db, hydra.launcher

// Elements (that may be swept over).

value: element | simpleChoiceSweep;

element:
primitive
| listValue
| dictValue
| function
;

simpleChoiceSweep:
element (COMMA element)+ // value1,value2,value3
;

// Functions.

argName: ID EQUAL;
function: ID POPEN (argName? element (COMMA argName? element )* )? PCLOSE;

// Data structures.

listValue: BRACKET_OPEN // [], [1,2,3], [a,b,[1,2]]
(element(COMMA element)*)?
BRACKET_CLOSE;

dictValue: BRACE_OPEN (dictKeyValuePair (COMMA dictKeyValuePair)*)? BRACE_CLOSE; // {}, {a:10,b:20}
dictKeyValuePair: ID COLON element;

// Primitive types.

primitive:
QUOTED_VALUE // 'hello world', "hello world"
| ( ID // foo_10
| NULL // null, NULL
| INT // 0, 10, -20, 1_000_000
| FLOAT // 3.14, -20.0, 1e-1, -10e3
| BOOL // true, TrUe, false, False
| INTERPOLATION // ${foo.bar}, ${env:USER,me}
| UNQUOTED_CHAR // /, -, \, +, ., $, %, *, @
| COLON // :
| ESC // \\, \(, \), \[, \], \{, \}, \:, \=, \ , \\t, \,
| WS // whitespaces
)+;

Elements​

Key​

Key is the component before the =. A few examples:

foo.bar           # A config key
hydra/launcher # A config group
group@pkg # A config group assigned to the package pkg
group@pkg1:pkg2 # A config group changing the package from pkg1 to pkg2

Quoted values​

Hydra supports both double quotes and single quoted values. Quoted strings can accept any value between the quotes. To include a single quote in a single quoted string escape it : \'. Same for double quote in a double quoted string.

Double quotes
"hello there"
"escaped \"double quote\""
"1,2,3"
"{a:10} ${xyz}"
"'single quoted string'"
Single quotes
'hello there'
'escaped \'single quote\''
'1,2,3'
'{a:10} ${xyz}'
'"double quoted string"'

Whitespaces in unquoted values​

Unquoted Override values can contain non leading or trailing whitespaces. For example, msg=hello world is a legal override (key is msg and value is the string hello world). Normally, your shell will interpret values with whitespaces as being multiple parameters (key=a b would be interpreted as key=a and b). To prevent this you can quote them with a single quote. For example:

$ python my_app.py 'msg=hello world'

Note that trailing and leading whitespace are ignored, the above is equivalent to:

$ python my_app.py 'msg=    hello world    '

Escaped characters in unquoted values​

Some otherwise special characters may be included in unquoted values by escaping them with a \. These characters are: \()[]{}:=, \t (the last two ones being the whitespace and tab characters).

As an example, in the following dir is set to the string job{a=1,b=2,c=3}:

$ python my_app.py 'dir=job\{a\=1\,b\=2\,c\=3\}'

Primitives​

  • id : oompa10, loompa_12
  • null: null
  • int: 10, -20, 0, 1_000_000.
  • float: 3.14, -10e6, inf, -inf, nan.
  • bool: true, false
  • dot_path: foo.bar
  • interpolation: ${foo.bar}, ${env:USER,me}

Constants (null, true, false, inf, nan) are case insensitive.

important

Always single-quote interpolations in the shell.

Dictionaries and Lists​

Lists​

foo=[1,2,3]
nested=[a,[b,[c]]]

Dictionaries​

foo={a:10,b:20}
nested={a:10,b:{c:30,d:40}}

Dictionaries are merged, not assigned. The following example illustrates the point:

Input config
db:
driver: mysql
user: ???
pass: ???
db={user:root,pass:1234}
db:
driver: mysql
user: root
pass: 1234
important

Always single-quote overrides that contains dicts and lists in the shell.

Sweeper syntax​

A choice sweep is comma separated list with two or more elements:

key=a,b                       # Simple sweep: ChoiceSweep(a, b)
key="a,b","c,d" # Elements can be quoted strings, ChoiceSweep("a,b", "c,d")
key=[a,b],[c,d] # Elements can be real lists, ChoiceSweep([a,b], [c,d])
key={a:10, b:20},{c:30,d:40} # And dictionaries: ChoiceSweep({a:10, b:20}, {c:30,d:40})

More sweeping options are described in the Extended Grammar page.

important

You may need to quote your choice sweep in the shell.

Functions​

Hydra supports several functions in the command line. See the Extended Grammar page for more information.

Working with your shell​

All shells interprets command line inputs and may change what is passed to the process. A good way to determine what the shell is doing to your command is to echo it.

# bash output
$ echo foo_{a:10,b:20} ${HOME} [b,c]*
foo_a:10 foo_b:20 /home/omry build_helpers
$ echo 'foo_{a:10,b:20}' '${HOME}' '[b,c]*'
foo_{a:10,b:20} ${HOME} [b,c]*

If in doubt, quote a command line element with a single quote (').

If you want to pass quotes to Hydra in a shell quoted string, it's best to pass double quotes.

$ echo '"hello world"'
"hello world"

You can use some shell specific commands to change their behavior, but the cost will be that their behavior will change.

Bash​

You can disable braces expansion, filename generation (globing) and hist expansion. Please note that this will change your shell behavior for the current session.

$ set +o braceexpand -o noglob +o histexpand
$ echo key1={a:10,b:20} key2=${HOME} key=[b]*
key1={a:10,b:20} key2=/home/omry key=[b]*
# does not help with () though:
$ echo key=choice(a,b,c)
bash: syntax error near unexpected token '('
$ echo 'key=choice(a,b,c)'
key=choice(a,b,c)

Other shells​

Send a PR to add information about your favorite shell here.