CSV, Table (reader and writer)

KNIME Analytics Platform
1

I have approximately 1 billion records with 10 variables (column)
What would be a better option (writer/reader) between CSV and Table nodes
Which one would be faster.

Please suggest other better options (if any)

2

For this amount of data a database options would be the best one.

2 Likes
3

Besides using a database (which is highly recommended) a useful exchange format could be Parquet.

3 Likes
4

@izaychik63, @mlauber71 Thank You for your advice, I will try your suggestions and will report hopefully

5

Hi @rkagrawal,

Table Writer writes data in internal KNIME format so both writing and reading data should be faster than using CSV format. But considering your data size @izaychik63 and @mlauber71 suggestions are definitely worth checking out.

Br,
Ivan

2 Likes
6

@ipazin Table Write-Reader worked well!

1 Like
7

@rkagrawal glad to hear that!
Br,
Ivan

8

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.