problem running KNIME on a cluster via qsub (Sun Grid Engine)

KNIME Analytics Platform
#1

Hey there!

I am trying to run KNIME workflows on a cluster by submitting jobs to nodes using qsub. I have a local KNIME 2.11.3 folder and upon qsub submission, KNIME gets executed. However, around 10 seconds into operation (when loading Weka nodes) I get a fatal error and a crash. The head of the error log is as follows:

------------------------------

 A fatal error has been detected by the Java Runtime Environment:

  SIGSEGV (0xb) at pc=0x00002abbb993ac4c, pid=114327, tid=46988475459328

 JRE version: Java(TM) SE Runtime Environment (7.0_60-b19) (build 1.7.0_60-b19)
 Java VM: Java HotSpot(TM) 64-Bit Server VM (24.60-b09 mixed mode linux-amd64 compressed oops)
 Problematic frame:
 C  [libc.so.6+0x7fc4c]  cfree+0x1c

 Core dump written. Default location: /exports/eddie3_homes_local/pgrabows/core or core.114327

------------------------------

 

Do you know what is a possible culprit here ? I never encountered this and can't find anything on Google. I would really appreciate any hints.

 

 

Best,

Piotr

#2

Hi Piotr,

Not really sure how to debug this one, but my first suggestion would be to upgrade to the latest version of KNIME.

Best,

Jon

#3

I didn't upgrade KNIME, but I did manage to solve this problem. It was actually extremely easy, I had to ask the cluster for a node with 8GB of RAM minimum (-l h_vmem=8G). For some awkward reason, any value below that was giving me this crash.

It might be that it's a problem that was specific to our cluster system, but somebody might stumble upon this thread in the future...