I have a workflow which produces one CSV Table at the end. Currently I am doing everything manually, meaning: I run my workflow and at the end I write down the file name in a writer-node and I indicate in which folder i want to save it.
How can I automate it? For example, KNIME creates file names automatically and saves them all automatically in one file location. I have read your post here: KNIME spirits, at your service! – blog.statinfer.com, but it was not of much help.
Maybe you have a mini-example workflow for that?
Thank you very much in advance!
Thank you for your reply! It is a periodic task, but must not necessarily be done weekly or daily. It just runs multiple times once I want to run it. Let me provide you with an exact example:
For example, I have a list of variable, lets say “fruit” in a “Table creator” node. It includes: strawberry, banana, apple and orange. Then it passes this filter varibale into SQL database. The SQL database sees that it must retrieve the data only for strawberries first. So it selects three columns like: amount available, price, and the amount sold. Then my workflow starts: it calculates some math and it ends with a csv file: “sold_strwaberries.csv” and at the very end it should save it to a local folder with a specific .csv file name. Once it saved it, it must come back to bananas and do exactly the same but just for bananas and until the last variable (orange).
So lets say a desired name would be: “sold_ FRUIT. csv” - so it must enter a specific fruit name inside a .csv file.
Regarding saving, yes I understand what you mean: I would always save it locally. In a local computer folder.
No sure that this is the best approach but I solve quite similar problem in a variable loop
In my case I produce reports with predefined names containing only appropriately filtered data. Names and path are in table creator node in my case, but you can easily make these names generated by the workflow itself.
Hello @Milovanova ,
Please have a look the linked workflow. The approach is quite similar to the one proposed by @DmitryIvanov76 in previous post.
I provided a data with retail of fruits by client, then arranged by columns. If your data is arranged by rows you can create a wildcard variable for a row filter (…)
Thank you for your solution! Maybe you can replace Google Drive Connector with some ready- csv-examples? Because it is not possible to read it for me. The first two nodes worked, but not the “CSV Reader” I wish I could see your .csv examples exactly, so that I will be able to run your workflow til the end.
Kind regards,
Alina
Hi @Milovanova
Sorry for the misunderstood. I was not expecting from you to read the data as when you download the workflow the data is embedded in the node.
Ok. There is not a big difference to work in local. You just don’t need the Google connectors and work with the whole local path. Please find attached source CSV -you can download to your local drive-, lets say “C:\Users\Milovanova\Documents\KNIME\fruits_retail_data.csv” remove the connectors and paste the selected address.
Aiming to create a local Path for your output files, you will need to edit the Java code of the Path node in the picture and replace with the following text.
And I think this is all that will make the workflow to work in your local.
BR
PS.- Just a clarification about this:
The first two nodes worked, but not the “CSV Reader”
When you run the ‘Google Authentication’ and the ‘Google Drive Conector’ nodes, you are in fact connecting to your own Google Drive. However, the file reader cannot find the file because is in my Drive.
I agree @Daniel_Weikert , but I tried to keep it simple focusing on the caption of the challenge; and on the functional requirements described in post three, where there wasn’t either any mention related to folder hierarchy.
I asked intentionally about this specific matter in post number two as well.
I’ve realized that some questions in the Forum are iterations on related topics, so having a cared constructed workflow in KNIME Hub helps for fast answers in further questions with small additions or modifications. At some point any functionality can be included.
Thank you very much I think the Google Drive connectors make this example unfortunately way too complicated. I removed the file system connection port, but one part still remained connected with Google drive connector (see it is still connected to WRITE csv node). I would not need Google connector…besides i work in an Organization where Google drive is not allowed while working with data, because we all use DSVO data requirements. But it is nice that you mentioned it, because maybe one day i will use google drive from my own computer.
Can you please, if it is still possible, ignore all drivers connections?
I just want to have a very basic solution 1) read files locally - 2) run the workflow - 3) save files locally under the different names, based on the input. Thank you very much in advance! I think that will finally solve the issue.
Besides your .csv file is only one. How I see it: three files with the same data format are stored somewhere in my laptop locally. The reader takes the first file - goes through workflow - then saves it with a name, goes back again, takes the second .csv file - goes through workflow - then saves it with a name, goes back again, takes the third csv. file… this is what I have meant. but you just created one and for some reason you are trying to unpivot…it just makes my example waaay too complictaed. I really need very very basic solution.
Hi.
I’ve modified the workflow to work in your local Paths. You only have to edit the desired Output Folder in the ‘Table Creator’ node shaded in grey.
I also simplified the Data to be arranged in rows. In this way the filtering is a way more simple removing the complexity of REGEX column filter.
I think the Google Drive connectors make this example unfortunately way too complicated. I removed the file system connection port, but one part still remained connected with Google drive connector (see it is still connected to WRITE csv node). I would not need Google connector…besides i work in an Organization where Google drive is not allowed while working with data, because we all use DSVO data requirements. But it is nice that you mentioned it, because maybe one day i will use google drive from my own computer.
