Hello everyone
I am very new into Knime and I just built a flow to upload excel files into an Azure DB
Everything works fine except for the time it takes to do the upload. I left the flow running for almost an hour and it completed only 4% Is there anything I could do to fix this?
The data being updated is a 100mb excel with 400k rows and 69 columns.
Hi @varelapablo,
with “DB upload” you mean the db writer node - right?
Check the option for batch size and increase the value to 5000 or 500.
This option defines how many rows of data are written to the db at the same time. The default is 1 - which is really slow.
If this does not fix your problem - please give more details about your problem - best would be an example workflow.
Hello! Yes I was referring to the DB Writer node.
Currently the Batch Size is in 1000, and if I change to 5000 the same happens.
Hello varelapablo,
for loading larger amounts of data into Azure DB or SQL Server please try the DB Loader node.
Bye
Tobias
Thank you
I am now trying this one, but I face the following error.
Do you know how can I adapt the var types to my data base?
ERROR DB Loader 3:8 The input column type at index 1 is “VARCHAR”, the database column type is “NVARCHAR”.
ERROR DB Loader 3:8 The input column type at index 2 is “VARCHAR”, the database column type is “NVARCHAR”.
ERROR DB Loader 3:8 The input column type at index 3 is “VARCHAR”, the database column type is “NVARCHAR”.
ERROR DB Loader 3:8 The input column type at index 4 is “VARCHAR”, the database column type is “NVARCHAR”.
ERROR DB Loader 3:8 The input column type at index 5 is “VARCHAR”, the database column type is “NVARCHAR”
Hi,
can you try to adapt the default type mapping in the Microsoft SQL Server Connector node. To do so open the dialog and go to the Output Type Mapping tab and change the mapping from String → VARCHAR to String → NVARCHAR as shown below:
Bye
Tobias
This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.
