How to auto-import XML files in a workflow

[tazar]

I have this flow for extracting some info from a XML to a CSV.
Fortunately this works smooth, but I have 100 files and so far I have imported manually and appended to the same file (same xml structure and same exported columns).

What I would like is to auto-import the 100 files, to stop introducing them manually. Probably it is something very easy but I am new to KNIME. Thanks a lot, any idea how to do would help since I can study further on myself.

[bruno29a]

Hi @tazar , you can use a Loop in this case.

You can create a table that contains the path of the XML files. Alternatively, if they’re all in the same folder, you can use the List Files/Folders node which will return the path of all the files in the folder in a table.

You can then use the Table Row To Variable Loop which will loop through the table, and it will generate the path into a variable. You can then use that variable to point to your XML file in your XML Reader. You can also generate a file path for your CSV and do the same in the CSV Writer where you use the variable as file path to write.

Something that look like this:

[tazar]

Thanks a lot for your time @bruno29a.
Unfortunately I did not manage to make it work.
I tried to replicate your flow and guess the settings but:

1. my knime 4.4.1 does not find any Variable Expressions Node. And could not dive into what it does.
2. since I tried to make it work without this node, it is only contacanating the first file (from the bunch of 100), and repeating the first row (of the first file) 100 times.
   I guess that I have not correctly configured the XML Reader node since I cannot set the variable and input folder setting.
   
   
   
   If you could further coach me with some advice I would be grateful.
   I am not very familiar with the mechanism of the loops and variables, so I guess I will have hard time making this 100-file loop work.

ps. in the last node (end loop) I set the included variable as “path”.

Thanks a lot.

[bruno29a]

Hi @tazar , I did not set anything in my nodes, as I did not have any sample xml files to work with (hence why they’re all not set up - red status).

You can find the Variable Expressions here:

Variable Expressions

You can simply drag and drop the node from the page to your workflow, and your Knime will install whatever is necessary for you to have the Expressions extension (this will also give you the Column Expressions node).

The Variable Expressions is there to generate a destination path/file for the CSV Writer dynamically, otherwise it will write to the same file. If you want to write to the same CSV file, then you don’t need the Variable Expressions (do still install it though even if you are not using it in this workflow)

For the XML Reader, why are you using the deprecated version? Use the newest version from here:

XML Reader

Again just drag and drop to your workflow. It should be able to read the path generated by the List Files/Folders. If it does not, then you may need to convert the path to URI.

[tazar]

Thanks a lot again @bruno29a. That worked. Without expression because I appended all in the same file.
You saved me once again.
I have figured out the mechanism of loops and variables too. So your help is much appreciated.

[bruno29a]

You are most welcome @tazar , I’m glad it’s resolved.

Do get the Expressions extension though, they’re very powerful nodes and allow much flexibility