Hey guys! It’s taking a long time to load TXT file. I’m working with txt files containing thousands of information .txt files with more than 40 thousand lines. Files with a size of approximately 3.52GB. I need to input the file with the File Reader node (Complex Format) because I cannot consider the delimiter as it is an unstructured file in its information (it is a fiscal file). I’ve already configured the knime .config file according to some tips, however it is taking almost three minutes to load the information on the node and there are even larger files that take longer. Who can help to say what can be done to have more performance? All suggestions will be valid to evaluate and try to improve. Thanks.

Hi @izaychik63 , from the thread you shared: “streaming helps only in case of multiple chained nodes with streaming functionality (not case here)”, that’s exactly the point I was making, at least that’s what I believe also. It seems from that thread that you tried and it worked for you. As I recommended to @andymesmo , do try it to see if there is huge gain.

Performance in input

KNIME Analytics Platform

izaychik63 March 8, 2022, 4:34pm 7

@bruno29a, Just try to bring your attention to the trail below.

1 Like