smallpond.logical.node.DataSourceNode#

class smallpond.logical.node.DataSourceNode(ctx: Context, dataset: DataSet)#

All inputs of a logical plan are represented as DataSourceNode. It does not depend on any other node.

__init__(ctx: Context, dataset: DataSet) None#

Construct a DataSourceNode. See Node.__init__() to find comments on other parameters.

Parameters#

dataset

A DataSet instance serving as a input of the plan. Set to None to create a dummy data source.

Methods

__init__(ctx, dataset)

Construct a DataSourceNode.

add_perf_metrics(name, value)

create_task(*args, **kwargs)

get_perf_stats(name)

slim_copy()

task_factory(task_builder)

Attributes

enable_resource_boost

num_partitions