Weird performance issue: nodes become slow, but 'fixed' by upstream row operations

I’m not sure if this is related to the latest update, but I did not notice it until updating to v4.3 today.

Several times today, certain parts of my workflow have become very slow. All kinds of nodes, such as sorters, row filters, and text preprocessing operations, started taking several times longer to run than usual (for example, sorting by a single column might take 20 seconds, when it usually takes an instant). By accident, I found that performance returned to normal if I inserted any row-based node upstream, such as a row filter, row sampler or RowID node, even if these nodes did not change anything (i.e. the row filter or row sampler was configured so as not to remove any rows). Notably, these nodes themselves took a long time to run, but once they did, the downstream nodes ran smoothly again.

I have not yet been able to identify any triggering node or action, and have not been able to replicate the issue by running the same nodes again. So I can’t really offer any example for diagnosis. I’m offering this post in case anyone else is experiencing something similar, or in the hope that someone who understands the internals of KNIME might be able to guess what is going on.

1 Like

Hi @AngusVeitch1 that’s indeed weird. Any chance you can share the part of the workflow or something similar that we can reproduce locally? I can’t explain the behaviour with any of the changes we did with 4.3, but I’m happy to do some profiling and figure out where the performance is actually lost.

Thanks for reporting and sorry for the trouble.

I still haven’t managed to isolate a cause, but it kept happening until I copied the components of my workflow into a new workflow, which so far has behaved well. So it looks like a problem connected with my original workflow rather than the environment more generally.

As soon as I can isolate a smaller part of the workflow that triggers the problem, I will share it and let you know.

2 Likes

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.