smallpond.dataframe.Session.wait

Session.wait(*dfs: DataFrame)

Wait for all DataFrames to be computed.

Example

This can be used to wait for multiple outputs from a pipeline:

df = sp.read_parquet("input/*.parquet")
output1 = df.write_parquet("output1")
output2 = df.map("col1, col2").write_parquet("output2")
sp.wait(output1, output2)