Skip to main content
Version: 1.0

Submitit Launcher plugin

PyPI PyPI - License PyPI - Python Version PyPI - Downloads Example application Plugin source

The Submitit Launcher plugin provides a SLURM Launcher based on Submitit.

Installation#

pip install hydra-submitit-launcher --upgrade

Usage#

Once installed, add hydra/launcher=submitit_slurm to your command line. Alternatively, override hydra/launcher in your config:

defaults:  - hydra/launcher: submitit_slurm

Note that this plugin expects a valid environment in the target host. usually this means a shared file system between the launching host and the target host.

Submitit actually implements 2 different launchers: submitit_slurm to run on a SLURM cluster, and submitit_local for basic local tests.

You can discover the SLURM Launcher parameters with:

$ python your_app.py hydra/launcher=submitit_slurm --cfg hydra -p hydra.launcher
# @package hydra.launcher_target_: hydra_plugins.hydra_submitit_launcher.submitit_launcher.SlurmLaunchersubmitit_folder: ${hydra.sweep.dir}/.submitit/%jtimeout_min: 60cpus_per_task: 1gpus_per_node: 0tasks_per_node: 1mem_gb: 4nodes: 1name: ${hydra.job.name}partition: nullcomment: nullconstraint: nullexclude: nullsignal_delay_s: 120max_num_timeout: 0additional_parameters: {}array_parallelism: 256

Similarly, you can discover the local launcher parameters with:

$ python example/my_app.py hydra/launcher=submitit_local --cfg hydra -p hydra.launcher
# @package hydra.launcher_target_: hydra_plugins.hydra_submitit_launcher.submitit_launcher.LocalLaunchersubmitit_folder: ${hydra.sweep.dir}/.submitit/%jtimeout_min: 60cpus_per_task: 1gpus_per_node: 0tasks_per_node: 1mem_gb: 4nodes: 1name: ${hydra.job.name}

You can set all these parameters in your configuration file and/or override them in the commandline:

python foo.py --multirun hydra/launcher=submitit_slurm hydra.launcher.timeout_min=3

For more details, including descriptions for each parameter, check out the config file. You can also check the Submitit documentation.

Caution: use of --multirun is required for the launcher to be picked up.

Example#

An example application using this launcher is provided in the plugin repository.

Starting the app with python my_app.py task=1,2,3 --multirun (see Multi-run for details) will launch 3 executions (you can override the launcher to run locally for testing by adding hydra/launcher=submitit_local):

$ python my_app.py task=1,2,3 --multirun[HYDRA] Sweep output dir : multirun/2020-05-28/15-05-22[HYDRA]        #0 : task=1[HYDRA]        #1 : task=2[HYDRA]        #2 : task=3

You will be able to see the output of the app in the output dir:

$ tree.โ”œโ”€โ”€ 0โ”‚ย ย  โ””โ”€โ”€ my_app.logโ”œโ”€โ”€ 1โ”‚ย ย  โ””โ”€โ”€ my_app.logโ”œโ”€โ”€ 2โ”‚ย ย  โ””โ”€โ”€ my_app.logโ””โ”€โ”€ multirun.yaml
$ cat 0/my_app.log [2020-05-28 15:05:23,511][__main__][INFO] - Process ID 15887 executing task 1 ...[2020-05-28 15:05:24,514][submitit][INFO] - Job completed successfully