Flow variable as file name when writing CSV file in a loop

KNIME Analytics Platform
1

Hi,

I have a process flow which reads daily txt files in as follows;

List of Files -> Iterate List of Files

In the Iterate Loop I have

Variable Based File Reader (a) -> [some actions] -> Concatenate (so now a single table) -> CSV Writer (b) ...   Loop End

In order to have dynamic name I also have Java Edit Variable (Simple) from the Flow Variable port of (a) which follows s.roughley's suggestion in this thread http://tech.knime.org/forum/knime-users/writing-multiple-files

where I'm replacing the .txt read in with .csv

and i can see in the snippet output it has worked.

In the CSV writer node I can select 'Output File Locaion:' via the 'v=' button as Flow Variable, and I can select the filename flow variable I've created in the java snippet, however I get two issues;

If the actual text box is empty (even after selecting the variable) I'm unable to leave the configure menu with a "Invalid Settings: Missing Output file name error".

If put any text in the text box, and then execute, I get an "Execute failed xecute failed: Unable to create directory for specified output file: [some path I've not entered such as the workspace] [the path from the flow variable]". So it looks like the CSV node is concatenating the variable I've provided with [some path], rather than just using my flow varibles path.

So, something I'm doing I guess - how can I correctly reference the flow variable created by the java snippet in the csv writter node so that the it uses this as the filename?

Cheers,

Andy

                           

2

The dialog doesn't like it when you specify a flow variable as the first action for the setting. It doesn't appear to populate the setting and the configure method therefore doesn't believe you have made a valid selection. 

 

You can just enter any random test before choosing your flow variable to work around this. It hasn't concatenated the values in the past for me. 

 

Have you tried: enter random text, select flow variable, select ok, reopen dialog to check the settings?

 

Cheers

 

Sam

 

3

Also worth checking whether the folder your variable is pointing too exists?  If it might not, you might want to add something like:

File f1 = new File(myPath);
File f2 = new File(f1.getParent());
Boolean success = f2.mkdirs();

to the edit variable node - or you can create the folder in a separate edit variable node before your loop starts if all the files are destined for the same folder, and then use the variable created as part of the file path creation within the loop - something like:

String myPath="C:\\My Folder\\"; //Note the \ are escaped!
File f = new File(myPath);
Boolean success = f.mkdirs();
return myPath;

Beware too - a slightly strange behaviour is that the folder will be created at the point at which the node is configured if you go for the latter option, not the point at which it is actually executed (when the variable will then be populated with the value!)

Anyhow, glad my original post has provided some inspiration!

Steve