Hi,
I don't understand why the "Table row to variable loop start" run very fast at the begin's run and along running, the running velocity slowed down when I go further.
Example : the first 10 minutes the node process 5000 Datas and the 10 next minutes, the node process only 1500 Datas
Is it a RAM problem ? I am not sure.
How can I improve the process velocity ?
Thanks for answer,
Thierry Crettol
Dear Thierry:
Can't tell without looking at a workflow. What do you do with the data in the workflow? Do you collect all of it at the end? Do you see a constant throughput if you discard all data within the loop and concatenate empty tables instead? (Just for the sake of diagnosing this issue.)
- Bernd
Hi,
Thank you for your answer.
I give you a workflow to show you what happens. To run the workflow you need to create :
- a database named "testKnime" and add the following table to mysql :
CREATE TABLE `feed` (
`id` int(11) NOT NULL,
`title` text NOT NULL,
`description` text NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
The result of the given workflow :
0 min : 0
4 min : 4110 rows processed
5 min : 4600 rows processed
6 min : 5156 rows processed
7 min : 5600 rows processed
8 min : 5980 rows processed
9 min : 6416 rows processed
10 min : 6800 rows processed
18 min : 9377 rows processed
So, you can find the workflow on this url :
http://ringtarget.com/workflow_group.zip
Thanks for your help and Happy new Year !!! :)
Thierry Crettol
It slows down from the "Variable to Table row" or from the "concatenate" Node. Beacause if I remove Column in the "Variable to Table row" Node it runs very faster !
It slows down from the "Variable to Table row" or from the "concatenate" Node. Beacause if I remove Column in the "Variable to Table row" Node it runs very faster !
The solution is maybe to use a ROWID colunn ...
And after concatenate with the row that match ROWID column
(Didn't prepare the DB but had a look at the workflow!)
If you have trouble with RowIDs then it might make sense to "Generate new RowIDs" in the Loop End node?
Hi,
Oki, thanks, I tried another way, using a ROW_ID key.
Instead of the Column appender, I made a new workflow with the Joiner node. It go faster.
See you.
Thierry Crettol