I'm downloading lots of data from a database into csv files one iteration at a time (one week of data at a time actually) and I'm noticing that the iterations get slower and slower every time. Up to the point where it literally takes hours for one iteration... and I'm only half way there!
I'm wondering if it's the heap space (which I've already set to 2048MB) which just can't keep up with the downloads. Or is this simply server-side and there's nothing I can do about it.
If it's my first hypothesis, is there a way to clear my heap space every 4-5 iterations (in a java node for example)?
Thanks so much for your help!