knio.Table.from_pandas Batch Size Error

[dvase]

Hi,
I get a batch size error in knio.Table.from_pandas

Code:

import knime.scripting.io as knio
import pandas as pd

cis_data_list = [v for k, v in knio.flow_variables.items() if k.startswith("Row") and not k.startswith("RowID")]

data = knio.input_tables[0].to_pandas()
df_list = [data[data['FIRINA_GIREN_ARAC_CIS_1+First(tag_value)'] == cis] for cis in cis_data_list]

df_concat = pd.concat(df_list, ignore_index=True)
print("ok")

knio.output_tables[0] = knio.Table.from_pandas(df_concat)

Error:

ok
❌ ValueError: all batches of the table must have the same size, but batch 1 has size 38 (expected: 2433)
Traceback (most recent call last):
  File "<string>", line 13, in <module>
  File "C:\Program Files\Knime_5.2.3\plugins\org.knime.python3_5.2.0.v202311291009\src\main\python\knime\api\table.py", line 381, in from_pandas
    return _backend.create_table_from_pandas(data, sentinel, row_ids=row_ids)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Knime_5.2.3\plugins\org.knime.python3.arrow_5.2.0.v202311290857\src\main\python\knime\_arrow\_table.py", line 165, in create_table_from_pandas
    return _create_table_from_pandas(data, sentinel, row_ids)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Knime_5.2.3\plugins\org.knime.python3.arrow_5.2.0.v202311290857\src\main\python\knime\_arrow\_table.py", line 151, in _create_table_from_pandas
    return _create_table_from_pyarrow(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Knime_5.2.3\plugins\org.knime.python3.arrow_5.2.0.v202311290857\src\main\python\knime\_arrow\_table.py", line 66, in _create_table_from_pyarrow
    _check_batch_sizes_constant(data)
  File "C:\Program Files\Knime_5.2.3\plugins\org.knime.python3.arrow_5.2.0.v202311290857\src\main\python\knime\_arrow\_table.py", line 111, in _check_batch_sizes_constant
    raise ValueError(

What should I do?

[mwiegand]

Hi @dvase,

with so little details (no screenshot, no example data nor a description of what you want to accomplish), it is quite challenging to provide any assistance. Any chance you can share more details please?

Best
Mike