Writing Excel Sheets to Database in a Loop

Hi,

As an extension to this reading all sheets…
I have an excel file with 3 different sheet names (test1,test2 and test3) which are also my table names. I want to insert the data from each sheet to a different db tables. All three sheets have 2 varchar & 1 date column.
It’s reading all the excel sheets but inserting only the first table. Then the looping throws an error “included column list contains invalid column(s)!”
In the DB writer, I have set the flow variable ‘Table’ as sheet. Looks like its taking the correct table but the column names are not matching with the table.
Please help.

Thanks
Venkata

You will need two loops nested within each other, one for the Excel files, one for the sheets. These two entries might help you set it up.

Excel Sheets iterate

Excel types of loops

Hi,

Thanks for your reply. Looks like I didn’t articulate my problem correctly. I am using only one excel with multiple sheets and I am able to read all of them. First row of all the three sheets is the column header for the corresponding db table field names. When I try to insert using DB writer, I am able to insert the first sheet into the first table properly. But when the loop progresses, in the second iteration it fails. It takes the table name correctly (sheet two’s name - test2), as I have set it as a flow variable (table name -> Shee) but the column names are still showing table one’s column names. It’s not changing dynamically.

I’ve attached the excel and .knwf files for your reference.

Please help.

Thanks
Venkata

OK you need a DB Table Creator node I think.

kn_forum_loop_excel_sheet.knwf (28.8 KB)

Thanks for looking into it.

But it doesn’t work. The problem stays the same. Records are not getting inserted into the tables.

Tables already exists, so there is no need for Table creator node. Only insertion needs to happen dynamically.

OK I think you should try the DB Insert node. In this example I first create empty tables and then fill them with the date. You would have to see if the column types would match.

Thanks @ mlauber71

I just stumbled across a similar post and found out the issue. The version that I am using is 4.0.2 and this problem was reported as a bug and it was fixed in version 4.1

I am going to upgrade the version and try it again. Thanks a lot for putting in so much effort to address the problem.

Thanks

2 Likes

Hi there @VenkataK,

think I remember this issue and it should be fixed with newer KNIME versions. Give it a try and if still something not working drop us a message here.

Welcome to KNIME Community!

Br,
Ivan

1 Like

Hi,

I upgraded the version and the flow is working fine now. Thanks @mlauber71 and @ipazin

@ipazin, to keep you posted, I upgraded it to Knime Analytics Platform 4.2.0 and in the last 30 mts it has crashed 3 - 4 times and sometimes the tool just hangs and doesn’t respond. I was only trying to change some of the nodes from deprecated to the equivalent new component.

Please suggest, which latest stable version (4.1.x) I should download.

Thanks
Venkata

1 Like

Hi @VenkataK,

glad it works.

Hmmm. First one to complain about new, 4.2.0, version crashing. Can you open a new topic for it? Then someone will address it and if other users experience it as well they can join easily. Please try to add as much as possible info such as which version you were using before, log file after crash, workflow example which is crashing, OS you are running, action you were preforming prior to crash and any other specific info that might help. Tnx!

Br,
Ivan

2 Likes

@ipazin looks like it’s the problem with my laptop configuration. I have been using i3 processor with 4GB RAM which works well for version 4.0 but crashes/hangs for the latest version 4.2. I tried using i5 processor laptop with 8GB RAM and it works fine.

So, I will not open a new thread for this issue.

Thanks

1 Like

Hi @VenkataK,

glad you found a solution. Indeed i3 with 4GBs sounds a bit too low…

Br,
Ivan

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.