DB Reader failed to execute: SAP DBTech JDBC: cannot allocate enough memory

[hueyyng]

Can anyone help with this? Not sure why the DB reader is not able to execute, is there any limitation on the number of lines it can handle?

In my query, I tried to limit to the top 10 lines to load but still showed the error message “Cannot allocate enough memory”, even in preview. Also, notice it has the default SELECT * FROM in the statement, I am scared by default it will pull overall before reading the custom query, has anybody have any idea how to fix this? Appreciate your help!

[wkhan]

Hi @hueyyng

Welcome to the forum. The DB Reader does not have a limit. I would add the DB Row Sampling – KNIME Hub node in between the DB Table Selector and DB Reader to add the Limit of 10. If you still get memory errors you can try writing the data to disk and seeing what is actually coming back by going to the “Job Manager Selection” Tab on the DB Reader and selecting Disk instead of Memory.

If that works then you can increase the memory settings for you KNIME AP or keep it on disk.

Regards,
Wali Khan

[hueyyng]

Thanks for getting back!
First trial with DB row sampling by pulling just 10 records - getting the same error.

Writing to Disk instead of Memory (with and without DB Row Sampling node) also gives me the same issue - sharing the error code below.

ERROR DB Reader 0:101:71 Execute failed: SAP DBTech JDBC: [4]: cannot allocate enough memory: [9] Memory allocation failed;exception 1000002: Allocation failed ; $size$=67108864; $name$=JEStep1; $type$=pool; $inuse_count$=79; $allocated_size$=16012530888; $failure_type$=STATEMENT_MEMORY_LIMIT; $failure_flag$=
,Exception in executor plan36909234@fmsaphdbnbp5a:31003 while executing pop 5

[hueyyng]

Hi @wkhan, appreciate your guidance on what can I do to enable it. Much appreciated!

[wkhan]

Hi @hueyyng

Looks like the error is coming from SAP and not KNIME with a type of STATEMENT_MEMORY_LIMIT

Regards,
Wali Khan

[hueyyng]

Hi @wkhan ,

Have you encountered this before or any way I can troubleshoot the issue? Not sure if any node can bypass the instruction to read the entire table before instructing the top 10? Thinking it shouldn’t be a problem if just pull a minimal record.

Best Regards,
Huey Yng

[wkhan]

Hi @hueyyng

The query does not execute until you use the DB Reader Node - none of the nodes should try to get the full table unless you have not put any limits on the DB Reader Node - so the limit is still causing SAP to return the memory limit. You can try to execute the query outside of KNIME, or check with the SAP team to see where the memory limit is coming from.

Regards,
Wali Khan

[hueyyng]

Hi @wkhan ,

Thank you! I managed to pull the data by having the DB Table Selector node pull just a subset of data that I just need, happy to see it is working!

Thanks for your time on the advice, appreciate it!

Best Regards,
Huey Yng

[wkhan]

Glad to see its working!