Cluster specs
Cluster specsΒΆ
dask-ctl can create Dask clusters for you from spec files.
These files describe the Python cluster manager which should be used along with any arguments.
# /path/to/spec.yaml
version: 1
module: "dask.distributed"
class: "LocalCluster"
args: []
kwargs:
n_workers: 2
threads_per_worker: 1
memory_limit: '1GB'
You can then create the cluster from the command line.
$ dask cluster create -f /path/to/spec.yaml
Or using the Python API.
from dask_ctl import create_cluster
cluster = create_cluster("/path/to/spec.yaml")
Both of these examples are equivalent to running the following Python code.
from dask.distributed import LocalCluster
cluster = LocalCluster(n_workers=2, threads_per_worker=1, memory_limit='1GB')