Loop Nodes to Return Intermediate Table Results

[Edlueze]

How do I preserve intermediate table data within a loop?

I am developing a new pair of Loop Nodes: (1) Tuning Loop Start, and (2) Tuning Loop End.

The Tuning Loop changes a set of input parameters (x) and checks whether an Objective Function flow variable (y) is being optimized (that is, maximized or minimized). The Tuning Loop will end by returning all of the intermediate calculations as well as the optimized x and y results. This works fine - so far so good.

I’ve decided to extend the node by providing another table output to the Loop End node. The new Loop End table output is supposed to contain the data from an input table at an intermediate point when the Objective Function is optimized.

It’s easy enough to output the last data set from an input table at the point the Tuning Loop ends. I just need to add this code to the end of the Loop End execute():

BufferedDataTable outputTableData = exec.createWrappedTable( (BufferedDataTable) inData[INPUT_TABLE] );
return new BufferedDataTable[]{ outputTuningParameters, outputIterationDetail, outputTableData }

But the same use of exec.createWrappedTable() doesn’t work when trying to preserve intermediate table data at the point the Objective Function was optimized. For example:

if( changedBestSolution ) {
	outputTableData = exec.createWrappedTable( (BufferedDataTable) inData[INPUT_TABLE] );
}

I’m looking for a way to clone the table dataset or preserve an intermediate table within a loop. How would I do that?

Note that the Loop End node is designed to capture the intermediate data from any input table within the loop. Hence no table specification is being defined.

[Edlueze]

This can be boiled down to a very simple question:

What is the best way to clone an input table?

[Edlueze]

It turns out that the answer was pretty easy. I was worried that a deep cloning routine was required for each data cell across all data types.

A new BufferedDataContainer needs to be created for the output table and defined outside of execute(). When an improvement is found, the execute() routine then needs to iterate over each row in the input table and add it to the new output table. This BufferedDataContainer will be preserved across the loop even though the ExecutionContext is changing. A WrappedTable won’t work as the underlying table seems to be removed each loop when the ExecutionContext changes.

//Clone the Input Table at the Optimization Point within the Tuning Loop so it can be passed through to the Output
if( changedBestSolution ) {
	m_outputTable = exec.createDataContainer( inData[INPUT_PORT_TABLE].getSpec() );
	RowIterator iteratorInputTable = inData[INPUT_PORT_TABLE].iterator();
	while( iteratorInputTable.hasNext() ) {
		DataRow rowInputTable = iteratorInputTable.next();
		m_outputTable.addRowToTable(rowInputTable);
	}
}