In KNIME is there anyway to record the history of results in running a workflow. For example, if I run a workflow and get out a set of results, I then decide to act on these results and modify the input value at the start of the workflow and rerun the workflow to get a new set of results, again based on these results I modify the input value a third time at the start and run the workflow a third time, and so on. Is there anyway in which KNIME can collate all the results from these iterations together, I cannot think of a way to do this. Ideally I need a collating node like a "Loop End" but without needing a "Loop Start" node. Any ideas ?
Thanks for directing me to the recursive loop post but its not quite what I am after.
What I am looking to do essentially is to input a Chemistry Molecule at the start (but it can be anything,a number for example) and this is fed into a predictor model which gives a set of results in the output. Based on these results, I then want to manually change the Molecule (or number etc) according to what I have learnt from the model results and rerun the workflow. Again based on these results, I go back and manually change the Molecule (or number) again and rerun the workflow a third time. I keep doing this until I am satisfied with the outcome. Now of course I can do all of that now, it just a straight forward workflow, but what I ideally want is all the iterative changes I made inbetween the first iteration and last iteration to be remembered in the output so I can see the intermediary Molecules (or numbers) and what their predictions were.
Ideally what is needed is some sort of History node, which collates every single result you run through a workflow until you reset it. Alternatively, another option would be to have an option in a writer node, such as XLS Output node, which has an option to append results to file, rather than just the overwrite file option.
How would you change the input manually given the output of the current loop iteration? I know about different Sketcher nodes, the Table Creator or even the Java Snippet that might become handy to manually manipulate your data. Each loop needs to be closed by a Loop End node which collects the results of each iteration. I still believe it's possible to perform this task. You can even include the result of each iteration into the next loop iteration by bypassing the Loop End with a CSV or Table Writer node and read in the data after the Loop Start. The Reader needs to be connected with a variable coming from the Loop Start in order to be reset during each iteration. Btw, the CSV Writer has an option to append data; the append option for the XLS Writer is currently work in progress.
I'm still confused on how to do this,sorry!. So what I do is draw one molecule in a chemical sketcher such as MarvinSketch and then run this through some chemistry nodes to get some predicted data, so predicted metabolism for instance. If the metabolism is say 90%, then I go back to the chemical sketcher and change the structure of the molecule in the region which I think may be causing high metabolism and then run the workflow again to get a new prediction on metabolism which now might be 70%. Again, I go back to the chemical sketcher and modify the molecule further which changes the metabolism again. So just running the workflow like this I just get the last result from the last molecule I drew, but I want to see the history of each molecule and its result as I go along as it helps in determining what to try next. If I try to use a looping system, then I am unsure how it lets me make manual alterations to the molecule as once you run the loop it keeps running and running until the loopcount is complete,so I would need to provide all the molecules upfront which is not possible without seeing the data to make the decision on what to try next. I dont see a way of getting a loop to wait until you make a manual change to the input after each loop iteration unless I am missing something.
If it is possible, then I am very excited to implement it.
Ahaha, I have managed to do it using what you said about the CSV Writer. I didnt need looping nodes in the end. I just wrote it to a CSV File in C:\Windows\Temp with Append enabled, and then straight after this had a CSV File Reader to read the file back in again, and connecting the CSV Writer and Reader by a Variable pipe to make the Reader reset for each iteration. Then after its read back in, I can then view the collated results altogether or export them back into Excel which gets round the current problem of no append option in the Excel Writer.