Number of rows able to be processed?

clint_r · August 10, 2018, 4:44am

Hi, I’m new to KNIME and was wondering if there was a row or column number limit for tabulated data? I have c. 3 M rows of data from a raster image that I was wanting to process.

Thanks.

Clint

ipazin · August 10, 2018, 7:05am

Hi!

You mean 3 million rows? I do manipulations and transformations with tables that have more then 30 million rows and 50 columns so I would say yes but the execution time will depend on your machine specification, Knime configuration,type of data you have in table and (mostly) on operations you preform with you data

Good luck!

Br,
Ivan

clint_r · August 10, 2018, 9:55am

Hi Ivan,

Thanks for your answer.

Yes that is correct. 3 million rows.
I’m interested in fuzzy clustering of 3 different bands extracted from a raster image.

Thanks again.

Clint

ipazin · August 10, 2018, 10:00am

Hi Clint!

Can’t help you on that one. Not familiar with fuzzy clustering nor how complex that method/analysis is.

Br,
Ivan

Uchechukwu1 · July 27, 2022, 4:22pm

Hello There,

I am interested in a node that could limit data stream to a number, say TOP1, example N=1.

Best,
Yu

ArjenEX · July 27, 2022, 4:36pm

Hi Yu,

You can use a regular Row Filter for this.

If the row limits are determined dynamically, use the flow variables to pass along those values.

Uchechukwu1 · July 27, 2022, 5:18pm

Hello ArjenEX. The regular row filter worked great for this.

Thanks alot.