Nevergrad is a derivative-free optimization platform providing a library of state-of-the-art algorithms for hyperparameter search. This plugin provides Hydra applications a mechanism to use Nevergrad algorithms to optimize experiment/application parameters.
Once installed, add
hydra/sweeper=nevergrad to your command. Alternatively, override
hydra/sweeper in your config:
The default configuration is here.
We include an example of how to use this plugin. The file
example/my_app.py implements an example of minimizing a (dummy) function using a mixture of continuous and discrete parameters.
You can discover the Nevergrad sweeper parameters with:
The function decorated with
@hydra.main() returns a float which we want to minimize, the minimum is 0 and reached for:
To run hyperparameter search and look for the best parameters for this function, clone the code and run the following command in the
You can also override the search space parametrization:
The initialization of the sweep and the first 5 evaluations (out of 100) look like this:
and the final 2 evaluations look like this:
The run also creates an
optimization_results.yaml file in your sweep folder with the parameters recommended by the optimizer:
Hydra provides a override parser that support rich syntax. More documentation can be found in (OverrideGrammer/Basic) and (OverrideGrammer/Extended). We recommend you go through them first before proceeding with this doc.
To override a field with choices:
You can tag an override with
ordered to indicate it's a
Choices are defined with a list in a config file.
Scalars can be defined in config files, with fields:
init: optional initial value
lower: optional lower bound
upper: optional upper bound
log: set to
truefor log distributed values
step: optional step size for looking for better parameters. In linear mode, this is an additive step; in logarithmic mode it is multiplicative.
integer: set to
truefor integers (favor floats over integers whenever possible)
upper bound will set the initial value to the middle of the range and the step to a sixth of the range.
Note: unbounded scalars (scalars with no upper and/or lower bounds) can only be defined through a config file.