Writing table rows as individual files

KNIME Analytics Platform
1

I would like to write each row in a table as an individual file into the same folder. The result is that all rows are written out as individual files. The file readers can read many files from a folder and then populate each file as an inidividual row in a table; I would like to do the same except writing the rows out as files (simple text files). Thank you!

2

You can use a Java Snippet:

// determine file name from Row ID
String fileName = "/your/path/" + ROWID + ".txt"; 
try (PrintStream ps = new PrintStream(new FileOutputStream(fileName))) {
	ps.print(c_yourColumn);
} catch (IOException e) {
	throw new IllegalStateException(
              "IOException when writing to " + fileName, e);
}

Alternatively, a loop using the Chunk Loop Start node, where you process row by row and pipe the filename to a CSV writer node through some flow variable should also work.

3

Thank you very much! I will give these ideas a try a report back if there are any hiccups. Thanks again!

4

It appears the Chunk Loop Start Node will only read one row in the variations avaliable. I will see if some other looping construct will work but if you have any other thoughts that would be appreciated. I would prefer to use this node as opposed to the Java snippet for various reasons. Thank you again!

5

Hi,

I made you an example, sometimes this is easier than many words :)

Let me know if I got your problem correctly. The workflow writes each row in one file in a temp folder the filename is rowid.csv.

Cheers,

Iris

6

Iris, 

First, thank you very much for your help! Getting this figured out will be a breakthrough for me! A few issues though which make the example unusable at this point:

1. After I run the example, I believe a temp folder, 'knime_tc', is supposed to be created and where the output of the CSV Writer will go. I do not find 'knime_tc' anywhere on my computer, though. When I look in the Configuration wizard for the CSV Writer, I see that the output is supposed to go to 'knime:\\kinime_mountpoint\test.csv'. I cannot find that location anywhere though I suspect that location is just used as a variable or temporary memory location prior to being put into 'knime_tc' (?). However, I also cannot find the test.csv file aywhere. So, not sure what is happening but I would love to see your nice example work! :)

2. When I load your example, I get what appears to be a benign error (though I cannot be certain). The error says your example was created in Knime 3.1 but I have 2.12. Can I (should I) upgrade to 3.1? If so, can I upgrade the existing instance or do I need to do a completely new install and import the workflows from the 2.12 workspaces? 

Thank you for your continued help! 

Paul

7

Hi Paul,

 

1. the temp folder ist created by the create temp dir node. If you open its outport the flow variable tem_path shows you the folder. There you find the files

1b) The path is read from a flow variable. You might see the warning " The filename parameter is controlled by a variable" in the configuration dialgo of the csv writer? This means that we will use a flow variable instead of the value written.

2.) If you don't use one of our servers updating to 3.1. will bring you quite some new features. However you need to reintall it as it will come with a new JAVA version under the hood.

But you can still use your old workspace, the workflows will all still run.

Best, Iris

 

8

Iris, thank you again for your help. I think we are getting there. Here is where I am:

		<p>I am able to expose the outport but I think I have to right-click the node and select &#39;Flwo variables with path information&#39; to get the value (?). When I do, I find this:&nbsp;/tmp/knime_Knime_Table_to_91244/knime_tc_sq4gjczu6hca. When I search my computer for this location, I cannot find it. I do see one similar folder but there is nothing in it. Either way, that file path format is very difficult to try to follow or use generally. I am not sure if I can set that path manually. Also, I do not see any values set in the Configuration wizard Flow Variables. Do I set the path there?&nbsp;</p>

		<p>I am not sure what you mean in 1(b). Could you clarify?&nbsp;</p>

		<p>Thank you!</p>
		</td>
	</tr>
</tbody>