Hi.
I have a workflow that takes a long time to do some data manipulation (2+ hours), due to data volumes involved and complexity of rules. At the end of this long data manipulation I have to do 2 things:
- truncate a db table
- rewrite the same db table with the new data.
This is done by 3 nodes in sequence:
1*) PostgreSQL Connector
2*) DB SQL Executor, which truncates the target db table
- DB Insert, which inserts the new data in the db table.
Now, the Connector and DB SQL Executor nodes have no predecessors, so they start as soon as I execute the workflow. As a consequence I have 2 problems:
- the target table remains empty for a long time
- when the DB Insert node receives the input data, the db connection is lost and the node execution hangs.
I suppose I have to add some predecessor in order to put the nodes 1*) and 2*) (see above) in the queue of the data manipulation. I tried with flow variables, but for some reason I’m not able to connect to the variable ports of the nodes 1*) and 2*).
I also tried to use the wait node but the “wait for time” / “wait to time” / “wait for file” methods are not suitable in my case.
Could anyone suggest a way to resolve this problem?
Thanks a lot