Importing txt files with differing columns

Hi All,

My first post so please be gentle with me.

I am trying to import a bunch of text files into Knime with the use of File Reader however it fails to do anything as some of the columns are not contained in all of the files:

I wished to select the columns I want to deal with in the next node (column filter) however as first step fails , there is no data to read the next one:

image

Any help would be appreciated, I would like to add some additional columns to the table once my txt file are read properly and I have the list of the final columns i want to perform further task on.

Thanks for reading.

1 Like

Hi @Pawel40Adi

Welcome to the KNIME Community!

To resolve this: go to the File Reader → Advances Settings. Then uncheck the option Fail if spec differs

As per the above, that way I’m able to read two files for which the first has two columns and the second one has three.

Hope this helps!

3 Likes

Thank you @ArjenEX , it does help indeed.However i have a feeling I am doing something incorrectly as i should be seeing around 150 columns in preview but can only see a few columns:

I think my delimiter is incorrect - I believe it should be ‘Tab’ rather than comma, but I am not sure how to select tab in configuration window. Autodetection does not resolve the issue.

Hi @Pawel40Adi

That’s something I unfortunately can’t really judge without seeing the actual files myself. If you could share a few files with the different format that contain a few lines of anonymized data that it will make life easier to help you.

I changed the delimiter from comma to tab in my test setup and with \t KNIME is able to read the files.

Also, as a general rule it’s always better to disable the feature to limit the preview to 10.000 rows as it often creates issues. It’s under Advanced Settings also.
image

1 Like

thank you @ArjenEX , i was able to use ‘\t’ to inform Knime I am looking for tab a delimiter (it took me long time to find that info online) - glad you also provided in in your post.

I’ll use data rows limit going forward.

thanks you

Hi @Pawel40Adi

Just to be sure, do not. Disable that feature since it’s frequently causing issues.

1 Like