I'm creating a new node which internal call an custom external cli tool. The tool then creates a csv file with a rowid column and the new data column. (This complicated flow is required because the library used is in C and 32-bit only).
Question is what is the most efficient way to append this new column? Using ColumnRearanger does not really seem to work in this scenario as far as I can tell. Only idea I had was to read the whole CSV into a Map and then use that Map to get the actual value. But then the whole csv will be in memory (should not really be a problem as this will anyway only be used on small tables). Still not ideal. Hwo could I read it line by line with Columnrearanger?
What are my other options just copy the table and append new column?