Parallel Chunk Loop fails with Python Script

[dlema]

Good day:

I have a very simple flow that has a Python Script node surrounded by a parallel chunk loop start and end as in the figure.

For sake of simplicity, the Python Script is just
output_table_1 = input_table_1.copy()
The process fails with error: “Execute failed: Runtime class of object “5.0” (index 69) in row “Row52348_?Row3_Row409_Row1141?” is Number (double) and does not comply with its supposed superclass Number (integer)”

As you know, Parallel Chunk Loop creates several equally defined Python Scripts and some of them returns column 69 as Number (integer) while others returns Number (double).

Any idea on how to solve this?

Thanks.

[mlauber71]

@dlema could this be because of the difference in the chunk of data that would be loaded into the python frame; and how they are processed. Could you just make sure that the columns would all have your preferred format in the end (.astype)

Converting data to a numeric type in Pandas

[dlema]

Well… the only issue is that it is a wide table, but with a little of patience, it can be done.

[mlauber71]

@dlema I think you can do it for types of columns at once. You would just have to collect them like this. Converting all Numbers to Float64:

my_columns_before= df.columns.to_series().groupby(df.dtypes).groups
print(my_columns_before)

numer_cols = df.select_dtypes(include='number').columns.tolist()
df[numer_cols] = df[num_cols].astype('float64')

my_columns_after = df.columns.to_series().groupby(df.dtypes).groups
print(my_columns_after)