The Ray Launcher plugin provides 2 launchers:
ray_aws launches jobs remotely on AWS and is built on top of Ray Autoscaler.
ray_local launches jobs on your local machine.
Once installed, add
hydra/launcher=ray_local to your command line. Alternatively, override
hydra/launcher in your config:
ray autoscaler expects a yaml file to provide configuration for the EC2 cluster; we've schematized the configs in
Discover ray_aws launcher's config
Upload & Download from remote cluster
If your application is dependent on multiple modules, you can configure
hydra.launcher.sync_up to upload dependency modules to the remote cluster.
You can also configure
hydra.launcher.sync_down to download output from remote cluster if needed. This functionality is built on top of
exclude is consistent with how it works in
Manage Cluster LifeCycle
You can manage the Ray EC2 cluster lifecycle by configuring the two flags provided by the plugin:
- Default setting (no need to specify on commandline): Delete cluster after job finishes remotely:
- Keep cluster running after jobs finishes remotely
- Power off EC2 instances without deletion
ray_local launcher lets you run
ray on your local machine. You can easily config how your jobs are executed by changing
ray_local launcher's configuration here
The example application starts a new ray cluster.
You can run the example application on your existing local ray cluster as well by overriding