Hi I am an Alteryx user looking to switch over to Knime and have some questions about file handling. In Alteryx, the best type of file for storing data in is the YXDB type file, and Alteryx does fairly well reading and writing these types of files. I noticed that in Knime, it seems that a CSV is the most common file type used for read and write. However, since I use databases with sometimes hundreds of millions of rows, storing this in a CSV is very inefficient. Also, I also noticed that the csv read speed is much slower in Knime than in Alteryx (8.6 seconds in Alteryx vs 44 seconds in Knime for a 157MB CSV). Furthermore, a 54MB YXDB in CSV form takes up 157MB. Are there any file types that KNIME handles that would a) speed up the read time and b) take up less space in storage than a CSV? Thank you for any feedback!

Best File Types for Knime

mlauber71 August 27, 2019, 5:53am 7

There is a possibility. You could use Hive external table function but there are some quirks as if now in KNIME.

1 Like