Issues importing multiple delimited files

Hi,

I’m trying to import multiple text delimited files into knime and write them to one Excel file, each text file being a separate tab in Excel. Problem is they are variable record types, though all follow the same delimiter. And some files can be blank. Running into different problems regardless of which method/nodes I try. When I try with file reader node I get either too few data elements or too many. I’ve tried cell splitter but run into issues when the file is blank. Hoping someone can help me out. Thanks.

Welcome to the forum @knimerdev.

Can you share some data and the workflow you’ve been creating?

1 Like

Hi @elsamuel - apologies for the long delay in responding to you. Below is my current workflow and a few sample files. If the file is empty I just want to either end the loop and go onto the next, or write it out to excel as an empty sheet. The files are all delimited in the same manner but could have different columns and/or be empty. In this example, when it gets to File 3 I get an error on Cell Splitter that you can’t have an empty table.

File 1.txt:
COLUMN1|COLUMN2
A|1
B|2
File 2.txt
COLUMN1|COLUMN2|COLUMN3
A|1|A1
B|2|B2
File 3.txt (file is empty)

It would’ve been easier if you had uploaded the workflow instead of posting a screenshot.

  1. Is there a reason that you’re converting the Path data to String and back again?
  2. What is the Column Filter node doing?

I have a simplified version of your workflow that accomplishes everything you mentioned, but using the File Reader node:

image

The Excel Writer is set to use the variable currentIteration as the sheet name, but this can be modified:

image

image

image

Can you upload an example data file that you’ve had trouble reading with the File Reader node?
The Cell Splitter node is not going to work on a blank table.

What variable did you use for File Reader? I’ve changed this so many times I’ve lost track so not sure why I converted Path to String and then back, perhaps because I couldn’t get the flow variable to work without it.

In the meantime I’ll look to see what I’m able to share due to company policies.

You can check out the workflow here:

elsamuel/Public – Importing Multiple Delimited Files – KNIME Hub

1 Like

Thank you, I think that worked! Not sure why I couldn’t get such a simple workflow to work for me. I did have to upgrade versions so maybe a previous version was missing some functionality (but more likely I just over complicated it).

2 Likes

Hi @knimerdev,
i’m not sure in which version the new Excel Writer was implemented, but in older Knime versions you only can use the Excel Writer for the first loop iteration, then you have to use the Excel Sheet Appender.

This means you need in addition a switch node and a endif node.

BR

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.