Source Data File size minimized after workflow executed

[catcat]

Hi All,

I have a data source file which contains 250K rows, CSV format, it is originally in rar format, after extract, the size is 44.1MB, I made some nodes to:

1. list files/folders–list the folder path of the file
2. files/folders meta info-- to get the file path
3. sorter- to sort the file creation date
4. row filter- to identify the file with most recent creation date
5. Table row to variable - to convert the file information to a “path”
6. CSV reader->read from local file system->file refer to the “path” → Advanced Setting-> unchecked the limit memory per column and limit data row scanned in CSV reader node.

But after executed the workflow the size is minimized from 44.1MB to 34KB… It appears in everytime after I executed the workflow, can anyone know where is the problem? (I am sorry that due to data security i cannot attach the csv here.)

[mlauber71]

@catcat it really is difficult to understand waht is happening. Things to consider.

How large is the data in its initial (compressed?) format. If the file might contain a lot of missings a CSV file might be shrunk to impressive (small) numbers.

Do you see the data within KNIME and are the number of lines the same as in the initial CSV. You can export data from within KNIME at every step of the way and check if you still have the right amout of data.

You might want to create an example with artificial data where your problem can be demonstrated. Yes that is some work but it will make it much easier to get help in the forum.

Reproducible (Minimal) Workflow Example

Hi there KNIMErs! What is this topic about? This topic is about helping yourself and KNIME community! How? By creating and sharing workflow examples that help and can be successfully ran by others When? When logic requires it. For example when you are opening a topic in order to get help (you are getting error or can’t configure node and would like someone else to take a look what is wrong; or you started a workflow and got stuck so need help with next step), when you are reporting …

[ipazin]

Hello @catcat,

are you saying that using CSV Reader changes file size afterwards file is read?

Br,
Ivan

[catcat]

Hi @mlauber71 , thank you very much for your kind assistance, It seems that my problem has been fixed, I wrote the analysis result into an excel file and the file location is directed by a path, seems the path somehow pointed to the data source file and overwrite it (don’t know why but it did so…).

[catcat]

Hi @ipazin , thank you very much for your kind assistance, It seems that my problem has been fixed, I wrote the analysis result into an excel file and the file location is directed by a path, seems the path somehow pointed to the data source file and overwrite it (don’t know why but it did so…).

[ipazin]

Glad to hear that @catcat!
Ivan