Hi all,
I am trying to read multiple files, extraxt some data and write SD files.
I first use the nodes List Files and Iterate List of Files, then do the desired modifications. This works fine. I want now to give different names of the output files and relate them to the original file, e.g., if the original filename is 'filename1'.smi i want to have an output file 'filename1'.sdf.
I tried the filename options in the SDF Writer and I either overwrite the original files or I get file names without the desired 'sdf' extension.
Thanks in advance for your hints.
Hi,
Inside the Iterate List of Files loop, insert a Rule Engine node and use the List Files variable (assume it will be called URL or something similar) as the default label and given the column a name such as Filename. Thus you will end up with a new column with the name of the original filename.
Now, after the loop, undertake all your necessary modifications and then use a String Replacer node on this Filename column and replace ".smi" for ".sdf" or use whatever transformation to get the end filename you desire.
Now connect to this string replacer node, a Groupby node to Group by this Filename Column with no aggregation columns set. This will now effectively give you a list of all the filenames. Now connect to this a Table Row to Variable Loop Start node.
Now connect also to the string replacer node, a Row Filter node. Right click on the Row Filter node and choose "Show Flow Variable Ports". Now connect the Table Row to Variable Loop Start node to the flow variable port of the Row Filter node with a red variable line. Inside the Row Filter node, choose to filter on this Filename variable.
Now connect a Loop End node to the Row Filter node, and additionally connect an SDF Writer node to the Row Filter node. In the flow variables tab of the SDF writer, from the dropdown next to Filename choose the Filename variable. Right click on the SDF Writer and choose show flow variables, do the same for the Loop End node. Now connect a red variable line from the SDF Writer right variable port to the left variable port of the Loop End.
Hopefully this will work for you.
Thanks a lot for your reply! I could not quite understand your protocol but it helped me finding out a solution. I start with a List Files node, then I use Rule Engine and String Replacer as you proposed, then I connected the String Replacer to Iterate List of Files. Inside the loop the Variable Based File Reader uses the original location to read the file, the SDF Writer however, uses the Filename as modified by String Replacer.
Once again, thanks a lot!
Alternatively:
Configure the file reader node (point it to any file for now - it doesnt matter which - you just need a valid file in the valid URL: box in the configuration dialogue. Now, click on the Flow variables tab, and in the right most box next to 'DataURL' (i.e. not the dropdown) type something - e.g. 'fname'. Now, whatever filename you have from your loop will appear as a flow variable called 'fname'. Close the configuration dialogue, and now right-click on the node again, and select 'Show flow variable ports'. Join the red blob at the top right to the input of a java edit variable node, and configure this to replace the input variable, return a string type, and with the snippet "return $${Sfname}$$.replaceAll(".smi",".sdf");"
You can now send this to the flow variable port on the top left of the sdf writer node, and set the filename to be the contents of the fname variable.
Thanks! That works prefectly!
Cheers