Loading validation set in DL python network learner node

Hi,
I’m trying to use the DL python network learner node for regression problem using tensorflow. I load the data as follows:

import tensorflow as tf
from TFModel import TFModel
import numpy as np

Get the training data from the table
x_train = input_table.iloc[0:77093, 1:63].values
y_train = input_table.iloc[0:77093, 0:1].values

#Get the validation data from the table
x_val = input_table.iloc[77094:, 1:63].values
y_val = input_table.iloc[77094:, 0:1].values

The above code loads x_train and y_train correctly. But I find that the x_val and y_val remains an empty array when executed. This leads to the validation loss over all epochs as 0. How can I resolve this ? I need to load the validation set and calculate the validation loss over epoch along with the training loss. Any suggestion is much appreciated.
I used this flow as reference to build mine: Train a MLP – KNIME Community Hub
Thanks in advance.

Hi
have you checked the length of your dataset? E.g. using .shape and see whether ther is a 77094 row?
br

Yes. There is a total of 90698 rows. out of which 77093 is given as training set and the rest as validation.

Hi @Sadhana_l,

what happens if you
print(input_table.iloc[77094:, 1:63].values)
print(input_table.iloc[77094:, 1:63])
print(input_table.iloc[77094:, 0:1].values)
print(input_table.iloc[77094:, 0:1])
?
Can you try to isolate the issue? e.g.
print(input_table.iloc[77094:78000, 1:63].values)
and then work your way through to
print(input_table.iloc[77094:90697, 1:63].values)
and see where it gets empty?

Best regards
Steffen

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.