Table Reader: Execute failed: Writing to table process threw "ClassCastException"

[mwiegand]

Hi,

I once raised this bug but it got closed before feedback about the “ClassCastException” issue was given. I want to bring this to your attention again as, when this error occurs, which it did again unexpectedly now w/o any particular change made, the whole execution and results must get discarded.

Though, I am able to limit the amount of rows read which indicates “garbage” data being written somewhere at the end, likely during the previous loop iteration, to the table.

ERROR Table Reader 6:1126:0:1018:0:1007 Execute failed: Writing to table process threw "ClassCastException"

Table Reader Caches Columns breaking workflow execution

Hi, I erratically face an issue I struggle to reproduce to help fixing it. Whilst I have a pretty good understanding and explanation to the circumstances, I still can not reproduce it in a workflow to share with you (but soon might have). The issue When a Table Reader initially reads a table, it seems to cache it’s columns as well as it’s column data types. Regardless if any transformation happens, the table reader, after the table being read was altered (column count, column type change), t…

Attached the Thread Dump. I have backed up the table file but can only share it privately due to client data being present.

231010-knime-thread-dump-ClassCastException.txt (330.3 KB)

I can share the table files, as done before, via a private download link. Please reach out to me per DM or email as usual.

PS: I have already isolated the data which triggers the issue and can reproduce it. I am currently trying to create a workflow with anatomized data which I can share.

PPS: It is caused by the JSON data. Albeit it’s valid JSON data, verified by Knime converting it to JSON and an external JSON validator, there is partially escaped JSON in the form of a string in a JSON key.

test-json-ClassCastException.txt (27.9 KB)

However, using that exact same setup in the test workflow I hereby share, it is NOT reproducible anymore. Applying the fallback from the other post linked at the top, replacing each character by itself, the table reader works (bcs it converts JSON to String). Reading the data also works, even when nothing got replaced, if the column type string is set.

Table Reader JSON triggers ClassCastException – mwiegand

Best
Mike

[armingrudd]

Hi @mwiegand,
I didn’t quite understand the issue here. I can execute the provided worklow without any issues. Both Table Reader nodes execute successfully.
I’m not sure if your issue is the truncated json or not. But we truncate long values to prevent long loading in port view. The data is not manipulated at all. You can test it with a JSON Path node after the Table Reader with JSON.

[mwiegand]

Hi @armingrudd,

yes, that’s what I mentioned as well that when copying nodes from one workflow to another, the issue is not reproducible. Only in the original workflow, the JSON string seems to cause the issue.

Best
Mike

[armingrudd]

I think I could reproduce the problem.

Would you check if the issue you have mentioned occurs while the workflow is running in columnar backend?

I didn’t get the same error message you mentioned and the Table Reader node doesn’t fail but it cannot read the JSON if the JSON is too long and the workflow is configured to columnar backend. I created a ticket for this issue: AP-21249

[mwiegand]

Glad to hear. How did you reproduced that rather elusive issue? The workflow runs with the current default row based backend.

PS: The test workflow and the workflow where it originated both use the row based backend.

[armingrudd]

What happens if you use the provided data here in the original workflow?
I mean, can you reproduce the issue if you change the data in the original workflow? (So you can share a workflow which reproduces the problem you mentioned).

@mwiegand

How did you reproduced that rather elusive issue?

Just used your workflow and tried to reproduce the problem. Thank you for helping.

[mwiegand]

Hi Armin,

seems I missed your reply back then and the issue resolved itself. However, it just came back and neither did anything changed on the data or workflow. I find no connection which could explain the cause or provide any hint. Here is a Thread Dump in case that helps.

Please note that I added the txt extension to upload the file:

240612 Table Reader ClassCastException - Ticket 73377 - threaddump-1718184682114.tdump.txt (150.4 KB)

Best
Mike

[mwiegand]

This is back but in a slightly altered variant just throwing

2024-10-25 08:41:41,703 ERROR Table Reader 4:1126:0:1018:0:1007 Execute failed: Writing to table process threw "ClassCastException"

When I skip the first line, as it’s the second iteration, the table is getting loaded indicting the structure is correct but the data isn’t.

Most intriguingly, when drag & dropping the table file from the explorer in Knime the file is getting read:

After the table got read once, I tried to pin down the cause as I start to believe it is not about the data but the path handling. Though, the formerly failing table reader was then able to read the data.

To recap all detail:

1. Location to file passed as Path variable
2. Data read wise during each iteration
3. Not persistently reproducible but failing occasionally at arbitrary points in time
4. Correlation presumed but not reproducible to the column data type JSON but not reproducible. Though, once converted to string, the issue occurred less often
5. If occurring, it is persistent requiring a full Knime restart
6. Table file shared to a Knime devoper was possible to read
7. If the presumably faulty line was skipped, the file could be read
8. Drag & Drop the table file to auto place a table reader with absolute location instead of relative to workflow area resolved the initially and constantly failing table reader node

@armingrudd can you see an value in that information?