We have a workflow which reads from a db, writes to tables on the server, and runs two parallel processes to read the tables and extract different data from them. The table reader nodes run at approximately the same time. We’ve found that about once a week one of the table reader nodes times out. It happens seemingly randomly, within one or the other of the two parallel processes, and usually on the same table reader node (which reads the largest table, about 433 MB).
The message provided is:
Table Reader 0:995:0:938
Read timed out
We’ve looked through the logs and cannot seem to identify a specific reason why this is happening. Just that it happens on the same workflow about once a week. I’m wondering if the ‘Connection timeout’ in the node configuration may play a role, or if we need to serialize the two processes instead of running them in parallel. I’ve run a basic test by creating two table reader nodes in a generic workflow and have them read the same table, starting each at various intervals apart to see if I could create a conflict, but the only result seems to be the second node takes a few seconds longer to complete than the first.
Any help would be appreciated.