Python snippet weired output_table after LSTM prediction

KNIME Extensions
1

Dear KNIME users,

I am using a Python Script (1 =>1) Node to make a prediction with a LSTM Keras neural network. After the algorithm finished the prediction I transform the nparray into an DataFrame so that I can copy it into the output_table.

For the transformation I am using pandas pd.DataFrame function.

After that I am printing the fist 5 rows with head() function and the console shows following output:

Prediction
0 162.258362
1 169.368301
2 133.968216
3 136.190872
4 147.283401

Anyways this happens within the node. If I want to execute the node directly in the workflow the output_table does not match with the result of my prediction.

KNIME_LSTM_output_table
KNIME_LSTM_output_table1115×700 33.7 KB

I am using output_table = pd_yhat.copy() to copy my prediction table to the output_table.

So my question is: Why does this happen? Does anyone have an idea why the internal print of the prediction table is okay and after I copied it to the output_table and execute the node it outputs such huge numbers with no relation to the real values?

Thank you very much in advance.

2

Hi lindig,

Do you use Flatbuffers to transfer the data from Python to KNIME? You can find that under File > Preferences > KNIME > Python > Serialization library.
There is a known issue with Flatbuffers in KNIME and float32 data (the usual data type used in neural networks).
Please find instructions on how to set up the alternative serialization library Arrow here.

This issue will be fixed in a future version of KNIME.

Marcel