Knime read csv speed

KNIME Analytics Platform
1

Hello,

I’m new on knime and used to work on alteryx.
I have huge difference of performance in reading a csv file between both tools.
My file is 20 million line, 9 columns (2 Go).
In alteryx, it takes 2 seconds to read it, in knime 2 minutes.
How can I improve the speed of knime for reading csv file ?

Thanks for your help

2

Hi @Landstalker,
Thank you for bringing this up. We had someone else report this just a few weeks ago and our developers are hard at work improving it. We have an internal ticket with ID AP-19698 that is already resolved and will make the CSV Reader parallelize the read, resulting in a read time of about 30 seconds for a 5GB CSV file. This improvement is scheduled to be released in KNIME 5.1, which will come out soon (I think July is realistic). I hope this will help you with your workflow!
And please keep the feedback coming, we appreciate it and are happy to improve KNIME AP based on it.
Kind regards,
Alexander

4 Likes
3

Thank you Alexander for your prompt reply. I will try it with the new version when it will be available.

Best regards

1 Like
4

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.