Optuna Sweeper plugin
This plugin enables Hydra applications to utilize Optuna for the optimization of the parameters of experiments.
#
Installation#
UsagePlease set hydra/sweeper
to optuna
in your config file.
Alternatively, add hydra/sweeper=optuna
option to your command line.
The default configuration is here.
#
ExampleWe include an example in this directory. example/sphere.py
implements a simple benchmark function to be minimized.
You can discover the Optuna sweeper parameters with:
The function decorated with @hydra.main()
returns a float which we want to minimize, the minimum is 0 and reached for:
To run optimization, clone the code and run the following command in the plugins/hydra_optuna_sweeper
directory:
You can also override the search space parametrization:
#
Search space configurationThis plugin supports Optuna's distributions to configure search spaces. They can be defined either through commandline override or config file.
#
Configuring through commandline overrideHydra provides a override parser that support rich syntax. Please refer to OverrideGrammer/Basic and OverrideGrammer/Extended for details.
#
Interval overrideBy default, interval
is converted to UniformDistribution
. You can use IntUniformDistribution
, LogUniformDistribution
or IntLogUniformDistribution
by casting the interval to int
and tagging it with log
.
Example for interval override
The output is as follows:
#
Range overriderange
is converted to IntUniformDistribution
. If you apply shuffle
to range
, CategoricalDistribution
is used instead.
Example for range override
The output is as follows:
#
Choice overridechoice
is converted to CategoricalDistribution
.
Example for choice override
The output is as follows:
#
Configuring through config file#
Int parametersint
parameters can be defined with the following fields:
type
:int
low
: lower boundhigh
: upper boundstep
: discretization step (optional)log
: iftrue
, space is converted to the log domain
If log
is false
, the parameter is mapped to IntUniformDistribution
. Otherwise, the parameter is mapped to IntLogUniformDistribution
. Please note that step
can not be set if log=true
.
#
Float parametersfloat
parameters can be defined with the following fields:
type
:float
low
: lower boundhigh
: upper boundstep
: discretization steplog
: iftrue
, space is converted to the log domain
If log
is false
, the parameter is mapped to UniformDistribution
or DiscreteUniformDistribution
depending on the presence or absence of the step
field, respectively. Otherwise, the parameter is mapped to LogUniformDistribution
. Please note that step
can not be set if log=true
.
#
Categorical parameterscategorical
parameters can be defined with the following fields:
type
:categorical
choices
: a list of parameter value candidates
The parameters are mapped to CategoricalDistribution
.