Hello!
I am trying to read csv or excel or copy/paste to table creator 160,000 rows. Nothing is working. How can I read an excel file with 160,000 rows into knime? Please help!
@IrynaK, copy/paste for this amount of data is a joke. As for Excel or CSV readers it should work. I load 10M rows from CSV in 10 min. KNIME may stop react after couple of node stops. Just close it and start over.
1 Like
Just drag and drop the file from your filemanager onto your KNIME cancvas
gr. Hans
Thank you. Just did. Get an error of Java heap space… Not sure what it is
Well, it give me an error saying cannot load the file. This is not the first time either.
As usual, Devil is in details. We need exact message, example of the records, WF, KNIME version.
2 Likes
There is no workflow, it is an Excel reader node only. 160K rows of data. Data is confidential so I cannot give you an example. There is no error except for when the node browser finds the file it does not populate the tab to chose and the preview says cannot load the file
Did you try to save file as CSV and load it? Can you load say 1K or less records from the file?
I did.
ERROR CSV Reader 2:229 Execute failed: New line in quoted string (or closing quote missing). In line 2.
See here for resolution ERROR CSV ReaderExecute failed: New line in quoted string (or closing quote missing)
1 Like
Had similar issue in the past (old laptop with 8gb RAM…if I remember correctly KNIME by default allocates 50% of your RAM…you can change this if you edit the config file somewhere). When you break up the file into smaller chunks and then read them in with Excel Reader / CSV reader node (40k or 60k rows at a time) and concatenate later in the work flow, it should work.
My issue was RAM issue. Recently upgraded to laptop with 32gb RAM and no such issues. I import at least 700k rows at a time (CSV and Excel). Hope this helps.
1 Like
When exporting from source to CSV consider using “|” (pipe) as the delimiter during export. Then tell the node that “|” (pipe) is the delimiter. I have found that commas tend to confuse most programs when importing a CSV file…especially when some of the quotes are missing. Open the CSV in Excel and save as CSV again, after you changed EXCEL’s delimiter from “,” (comma) to “|” (pipe). You can change the default delimiter in your regional settings. Only draw back is that any Excel formula (when working in Excel) will now require “|” (pipe) as separator in the formula…requires bit of getting used to. This is a work around that I have used in the past.
2 Likes
This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.