Impossible to run multiple instances of knime (Could not create metadata repository for:...)

Dear all,
I’m trying to wrap a number of knime workflows and launch them in parallel in a cluster. I got the following errors:

java.lang.IllegalStateException: Could not create metadata repository for: file:/users/bi/lcozzuto/.eclipse/611213136_linux_gtk_x86_64/configuration/org.eclipse.osgi/251/data/listener_1925729951/
at org.eclipse.equinox.internal.provisional.p2.directorywatcher.RepositoryListener.initializeMetadataRepository(RepositoryListener.java:112)
at org.eclipse.equinox.internal.provisional.p2.directorywatcher.RepositoryListener.(RepositoryListener.java:59)
at org.eclipse.equinox.internal.p2.reconciler.dropins.DropinsRepositoryListener.(DropinsRepositoryListener.java:82)
at org.eclipse.equinox.internal.p2.reconciler.dropins.Activator.watchDropins(Activator.java:540)
at org.eclipse.equinox.internal.p2.reconciler.dropins.Activator.start(Activator.java:173)
at org.eclipse.osgi.internal.framework.BundleContextImpl$3.run(BundleContextImpl.java:774)
at org.eclipse.osgi.internal.framework.BundleContextImpl$3.run(BundleContextImpl.java:1)
at java.security.AccessController.doPrivileged(Native Method)
at org.eclipse.osgi.internal.framework.BundleContextImpl.startActivator(BundleContextImpl.java:767)
… 12 more
Root exception:
java.lang.IllegalStateException: Could not create metadata repository for: file:/users/bi/lcozzuto/.eclipse/611213136_linux_gtk_x86_64/configuration/org.eclipse.osgi/251/data/listener_1925729951/
at org.eclipse.equinox.internal.provisional.p2.directorywatcher.RepositoryListener.initializeMetadataRepository(RepositoryListener.java:112)
at org.eclipse.equinox.internal.provisional.p2.directorywatcher.RepositoryListener.(RepositoryListener.java:59)
at org.eclipse.equinox.internal.p2.reconciler.dropins.DropinsRepositoryListener.(DropinsRepositoryListener.java:82)
at org.eclipse.equinox.internal.p2.reconciler.dropins.Activator.watchDropins(Activator.java:540)
at org.eclipse.equinox.internal.p2.reconciler.dropins.Activator.start(Activator.java:173)
at org.eclipse.osgi.internal.framework.BundleContextImpl$3.run(BundleContextImpl.java:774)
at org.eclipse.osgi.internal.framework.BundleContextImpl$3.run(BundleContextImpl.java:1)
at java.security.AccessController.doPrivileged(Native Method)
at org.eclipse.osgi.internal.framework.BundleContextImpl.startActivator(BundleContextImpl.java:767)
at org.eclipse.osgi.internal.framework.BundleContextImpl.start(BundleContextImpl.java:724)
at org.eclipse.osgi.internal.framework.EquinoxBundle.startWorker0(EquinoxBundle.java:932)
at org.eclipse.osgi.internal.framework.EquinoxBundle$EquinoxModule.startWorker(EquinoxBundle.java:309)
at org.eclipse.osgi.container.Module.doStart(Module.java:581)
at org.eclipse.osgi.container.Module.start(Module.java:449)
at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.incStartLevel(ModuleContainer.java:1620)
at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.incStartLevel(ModuleContainer.java:1600)
at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.doContainerStartLevel(ModuleContainer.java:1571)
at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.dispatchEvent(ModuleContainer.java:1514)
at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.dispatchEvent(ModuleContainer.java:1)
at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:340)
INFO main BatchExecutor ===== Executing workflow module_workflow_shotgun.knwf =====
WARN main WorkflowManager Unable to insert connection “[12(1) -> 9( 1)]”
WARN main WorkflowManager Unable to insert connection “[12(1) -> 10( 2)]”
WARN main Node Node can’t be executed - Node “Input File” not available (provided by “Freie Universitaet Berlin, Universitaet Tuebingen, and the GenericWorkflowNodes Team”; plugin “com.genericworkflownodes.knime” is installed)
WARN main Node Node can’t be executed - Node “Input File” not available (provided by “Freie Universitaet Berlin, Universitaet Tuebingen, and the GenericWorkflowNodes Team”; plugin “com.genericworkflownodes.knime” is installed)
WARN main Node Node can’t be executed - Node “Input File” not available (provided by “Freie Universitaet Berlin, Universitaet Tuebingen, and the GenericWorkflowNodes Team”; plugin “com.genericworkflownodes.knime” is installed)
WARN main Node Node can’t be executed - Node “Input File” not available (provided by “Freie Universitaet Berlin, Universitaet Tuebingen, and the GenericWorkflowNodes Team”; plugin “com.genericworkflownodes.knime” is installed)
WARN main Node Node can’t be executed - Node “Input File” not available (provided by “Freie Universitaet Berlin, Universitaet Tuebingen, and the GenericWorkflowNodes Team”; plugin “com.genericworkflownodes.knime” is installed)
WARN main Node Node can’t be executed - Node “Input File” not available (provided by “Freie Universitaet Berlin, Universitaet Tuebingen, and the GenericWorkflowNodes Team”; plugin “com.genericworkflownodes.knime” is installed)
WARN main Node Node can’t be executed - Node “Input File” not available (provided by “Freie Universitaet Berlin, Universitaet Tuebingen, and the GenericWorkflowNodes Team”; plugin “com.genericworkflownodes.knime” is installed)
WARN main Node Node can’t be executed - Node “Input File” not available (provided by “Freie Universitaet Berlin, Universitaet Tuebingen, and the GenericWorkflowNodes Team”; plugin “com.genericworkflownodes.knime” is installed)
WARN main Node Node can’t be executed - Node “Input File” not available (provided by “Freie Universitaet Berlin, Universitaet Tuebingen, and the GenericWorkflowNodes Team”; plugin “com.genericworkflownodes.knime” is installed)
INFO main BatchExecutor Workflow execution done Finished in 1 sec (1784ms)
INFO main BatchExecutor ========= Workflow did not execute sucessfully ============
Knime:
GTK+ Version Check

Command error:
Unable to init server: Could not connect: Connection refused
Knime: Cannot open display:
Knime:
JVM terminated. Exit code=4
/usr/local/knime_3.5.3//plugins/org.knime.binary.jre.linux.x86_64_1.8.0.152-01/jre/bin/java
-server
-Dsun.java2d.d3d=false
-Dosgi.classloader.lock=classname
-XX:+UnlockDiagnosticVMOptions
-XX:+UnsyncloadClass
-Dsun.net.client.defaultReadTimeout=0
-XX:CompileCommand=exclude,javax/swing/text/GlyphView,getBreakSpot
-Xmx2048m
-Dorg.eclipse.swt.internal.gtk.disablePrinting
-jar /usr/local/knime_3.5.3//plugins/org.eclipse.equinox.launcher_1.3.200.v20160318-1642.jar
-os linux
-ws gtk
-arch x86_64
-launcher /usr/local/knime_3.5.3/knime
-name Knime
–launcher.library /usr/local/knime_3.5.3//plugins/org.eclipse.equinox.launcher.gtk.linux.x86_64_1.1.400.v20160518-1444/eclipse_1617.so
-startup /usr/local/knime_3.5.3//plugins/org.eclipse.equinox.launcher_1.3.200.v20160318-1642.jar
–launcher.overrideVmargs
-exitdata 408027
-data /nfs/software/bi/biocore_tools/git/nextflow/Qcloud/work/1c/a4f416ae9666488e41d5ce1a532a91
-clean
-consoleLog
-application org.knime.product.KNIME_BATCH_APPLICATION
-reset
-nosave
-workflowFile=module_workflow_shotgun.knwf
-workflow.variable=input_mzml_file,QC01_180308_Q_QC1X_01_03.mzML,String
-workflow.variable=output_qcml_file,QC01_180308_Q_QC1X_01_03.qcml,String
-workflow.variable=output_featurexml_file,QC01_180308_Q_QC1X_01_03.featureXML,String
-workflow.variable=output_idxml_file,QC01_180308_Q_QC1X_01_03.idXML,String
-workflow.variable=input_fasta_file,shotgun_bsa.fasta,String
-workflow.variable=input_fasta_psq_file,shotgun_bsa.fasta.psq,String
-vm /usr/local/knime_3.5.3//plugins/org.knime.binary.jre.linux.x86_64_1.8.0.152-01/jre/bin/java
-vmargs
-server
-Dsun.java2d.d3d=false
-Dosgi.classloader.lock=classname
-XX:+UnlockDiagnosticVMOptions
-XX:+UnsyncloadClass
-Dsun.net.client.defaultReadTimeout=0
-XX:CompileCommand=exclude,javax/swing/text/GlyphView,getBreakSpot
-Xmx2048m
-Dorg.eclipse.swt.internal.gtk.disablePrinting
-jar /usr/local/knime_3.5.3//plugins/org.eclipse.equinox.launcher_1.3.200.v20160318-1642.jar

can you help me on this?

Thanks
Luca

Hi Luca,

how did you install the plugins on the cluster instance of KNIME?
Does it work if you only run one job (no parallelism)?

Best
Julianus

I have Knime installed in a singularity container. One job it works, more than one no.

It seems like it is not enough to only change the workspace. The metadata repository seems to be created based on different settings.
@thor Do you have any ideas on how to run multiple instances of KNIME so that they do not interfere with each other?

I haven’t heard of any problems running multiple instances of a single installation as long as they use different workspaces. I fact this frequently happens on any KNIME Server. However, in case the file system permissions in the directory mentioned in the error message are messed up then this could be the reason. This shouldn’t be a problem if only a single user is using that installation. If multiple users are using an installation the installation must be read-only to all of them otherwise they mess up permissions for each other.

Hi, I don’t know what are you referring, but jpfeuffer gave me the solution for setting new workspaces:

knime -data $DIRECTORY -clean -consoleLog -vmArgs -Duser.home="$DIRECTORY"

Many thanks

1 Like

So yes, we basically also changed the user.home which seems to be used for the p2 metadata folder.
If it was write permissions or file locks I don’t know. Seems to be solved.

2 Likes