Execute failed: Last batch index is 26040, but maximum batch index is 2277.

[TheRealUser]

I tried to Do the following in a Python (labs) node and it works without a problem. you can also display the Table inside Python. But when I try to use it in Knime again the next Node always gives me:
Execute failed: Last batch index is 26040, but maximum batch index is 2277.

data= knio.input_tables[0].to_pandas()
def r1lookup(df):
    
    r1=data.loc[data['ID'].isin(df["ID"])].loc[data["r"]==1]
    return r1

conbywer=data.groupby(["W"]).apply(r1lookup)
knio.output_tables[1] = knio.write_table(conbywer)

[mlauber71]

@TheRealUser welcome to the KNIME forum. Maybe you can share an example where this does happen.

Then you should check if it could be related to an index:

Error in Python script node

Hi @mohini1329, as the others have already suggested, you should make sure that each row in the Pandas DataFrame has a unique index. Dropping/resetting the index is not the best option though, because KNIME uses the RowID (which is equal to the DataFrame index) to identify rows when you select / highlight individual cells in your data. If you reset the index, then this row identification will no longer work. The best solution would be to let the index of all values in your input table as they…

[TheRealUser]

Thanks for the Response. I tried resetting the index and generating new one out of numbered Strings. Sadly neither worked.

[carstenhaubold]

Hi @TheRealUser,
sorry to hear that you’re running into such an error.

Could you provide us with a table with some dummy input data so we can reproduce the error? Does it already show up with a small table with a few rows only?

Resetting the index is a good idea, this would have to happen on the conbywer data frame right before passing it to knio.write_table.

[TheRealUser]

I now tried around a bit, but can’t really find the problem. The only thing I found out that I have a few Data errors in my dataset with ID and r together not being unique. if I remove those the error dispersers. I tried to generate some artificial data with and without duplicates but it worked both times independent of size. I didn’t really know what the problem is at this point, but with the cleaned data it works. Thanks for the help anyway.

[carstenhaubold]

Thanks for the feedback @TheRealUser. I’m glad to hear that cleaning up the data resolved the error.

We’ll have a look whether we can reproduce the error and will try to add more useful error/warning messages so it becomes easier to pinpoint the problem in the future.