Dear list members,
First of all, I'm a newbee of the KNIME package, but I already have a problem which I can't solve by reading and google-ing... :-(
I run into a memory problem when I try to cluster (fingerprint) my 180.000 ligands. The error I get is
ERROR Distance Matrix Calculate Execute failed: GC overhead limit exceeded
I already used "Write tables to disk" within the Memory Policy menu. No changes.
Any hints how to change the limits are highly appreciated!! Or even a hint that it is not possible to use such a large number of ligands...
I'm using KNIME 2.3.3, OS openSUSE 11.4 x86_64, Java is java-1_6_0-openjdk-184.108.40.206_b220.127.116.11-1.2.1.x86_64
Running into a similar issue on my end. I have created fingerprints of 200K molecules and I am running a distance matrix calculation on these as well. In my case I chose to 'write tables to disk' as the memory policy, problem is KNIME wrote to 120GB of disk space. I understand that each fingerprint is taking up 512 bytes, and I am calculating 200K rows by 300+ columns but I dont think that would result in 120 GB file.
One question I have, is it possible to change the location of where KNIME writes its temp files as I could mount a larger drive and brute force this through?
Thanks for any help,
Well, the distance matrix for 200,000 molecules has 200,000*200,000/2 entries. Given that each entry is a double value with 8 Bytes this gives 149GB. The compression brings it down to 120GB. You can change the temp location in the KNIME preferences (File->Preferences->KNIME).
Thank you for the response Thor, yup you are right about the size I did not multiply by 8 bytes per double. The ability to change the write location of the temp files is key, thank you so much I was getting a bit frustrated!