dask_find_or_start_client

HSTB.kluster.dask_helpers.dask_find_or_start_client(address=None, number_of_workers=None, threads_per_worker=None, memory_per_worker=None, multiprocessing=True, silent=False, logger=None)

Either start or return Dask client in local/networked cluster mode. Use this function whenever you need access to a new or existing cluster

Parameters

• address (Optional[str]) – ip address:port for existing or desired new dask server instance. Will accept anything that dask.distributed.get_client accepts.

• number_of_workers (Optional[int]) – integer number of workers in the LocalCluster, only used when address = None (local cluster)

• threads_per_worker (Optional[int]) – integer number of threads per worker in the LocalCluster, only used when address = None (local cluster)

• memory_per_worker (Optional[str]) – string representation of the memory allowed per worker, ex: ‘10GB’

• multiprocessing (bool) – if True, will allow multiple workers, if False, will only use threads

• silent (bool) – whether or not to print messages

• logger (Optional[Logger]) – if included, will print output messages to the provided logger

Returns

Client instance representing Local Cluster/Networked Cluster operations

Return type

dask.distributed.client.Client