smallpond.execution.task.RuntimeContext#

class smallpond.execution.task.RuntimeContext(job_id: JobId, job_time: datetime, data_root: str, *, num_executors: int = 1, random_seed: int | None = None, env_overrides: Dict[str, str] | None = None, bind_numa_node=False, enforce_memory_limit=False, max_usable_cpu_count: int = 1024, max_usable_gpu_count: int = 1024, max_usable_memory_size: int = 17592186044416, secs_wq_poll_interval: float = 0.5, secs_executor_probe_interval: float = 30, max_num_missed_probes: int = 6, fault_inject_prob=0.0, enable_profiling=False, enable_diagnostic_metrics=False, remove_empty_parquet=False, skip_task_with_empty_input=False, shared_log_root: str | None = None, console_log_level='INFO', file_log_level='DEBUG', disable_log_rotation=False, output_path: str | None = None)#

The configuration and state for a running job.

__init__(job_id: JobId, job_time: datetime, data_root: str, *, num_executors: int = 1, random_seed: int | None = None, env_overrides: Dict[str, str] | None = None, bind_numa_node=False, enforce_memory_limit=False, max_usable_cpu_count: int = 1024, max_usable_gpu_count: int = 1024, max_usable_memory_size: int = 17592186044416, secs_wq_poll_interval: float = 0.5, secs_executor_probe_interval: float = 30, max_num_missed_probes: int = 6, fault_inject_prob=0.0, enable_profiling=False, enable_diagnostic_metrics=False, remove_empty_parquet=False, skip_task_with_empty_input=False, shared_log_root: str | None = None, console_log_level='INFO', file_log_level='DEBUG', disable_log_rotation=False, output_path: str | None = None) → None#

Methods

`__init__`(job_id, job_time, data_root, *[, ...])
`cleanup`([remove_output_root])	Clean up the runtime directory.
`cleanup_root`()
`get_local_gpus`()
`initialize`(exec_id[, root_exist_ok, ...])
`new_task_id`()
`set_current_task`([task])

Attributes

`available_memory`
`exec_plan_path`
`job_root_dirname`
`job_status_path`
`logcial_plan_graph_path`
`logcial_plan_path`
`numa_node_count`
`physical_cpu_count`
`ray_log_path`
`runtime_ctx_path`
`sched_state_path`
`secs_executor_probe_timeout`
`task`
`total_memory`
`usable_cpu_count`
`usable_gpu_count`
`usable_memory_size`