Hello,
I’m a new Knime user. I am learning the Knime white paper Big data, Smart Energy, and Predictive Analytics (2013) under energy category, IoT section. The full data for this project is about 176 million rows. The workflow runs fine when using one sixth of the data. However, Knime stops responding when reading the full 176 million rows of data. The workflow is attached named prepare data
1-Prepare Data reduced reset.knwf (302.3 KB) .
Thus, I attempted to run one sixth of the data a time for six times, this worked. However, when I tried to concatenate the hourly data together, Knime did ok for reading and concatenate the data, but it stopped responding when writing the concatenated data to a csv file, workflow attached 1-2 concatenate_data.knwf (23.1 KB). This exact same workflow to concatenate data worked for a small amount of data. I wonder why Knime stops responding when dealing with large amounts of data. Is there something wrong with my Knime configuration or my laptop configuration? I’m using default Knime configuration. Thank you.
Regards,
Zhengnan