Hello community
I m taking data from microsodt sql server, and when i want to go back to db query reader, it takes ages to refetch data. i have to create abother db reader query for it to fetch meta data quickly.
is there smth i am missing?
is there a way to fix this please help
Hey there,
I used to query MS SQL Server in past projects - during development queries on smaller sets of test data performed OK initially, but when testing on larger datasets that reflected what we’d encounter in products things got really slow.
It turned out that the issue was not on KNIME node side, but on SQL side - we ended up having to assign “more compute power” (not sure anymore about the exact terminology…) and that fixed it for us…
1 Like
Hello @MartinDDDD do you mean you brought pc s with higher specs? For me, i think of it as limitations on the network not hardware
Yes and No: In azure cloud we had to add some sort of credits to have more computing resources allocated which ended up speeding up queries that were send from Knime nodes to SQL DB on Azure.
Issue we found was that the query processing on DB was the issue - not the Knime node - I.e. the knime node took so long because there was a demanding query being processed in the cloud
Edit: Found a post I made on a similar topic:
The term I couldn’t remember was Database Transaction Units (DTUs), that we had to increase to speed up query processing
I have used KNIME an MSSQL without much problem and with millions of lines and gigabytes of data without much problem…