Unable to load workflows after updating KNIME Server and its executor

rahul_kumar · January 3, 2021, 5:34pm

Hi,

I have updated KNIME Server from 4.10 to 4.12 and the executor from 4.1.4 to 4.3 but now my executor is not working and it says in webportal that no executor information is available.

Also when i try to run any job it say no RMI executor running

Below is my Server configuration file for executor where knime_exe is the soft link for knime 4.3 analytics platform

com.knime.server.executor.knime_exe=/opt/knime/knime-latest/knime
com.knime.server.executor.max_instances=
com.knime.server.executor.max_lifetime=1d
com.knime.server.executor.prestart=true
com.knime.server.executor.reject_future_workflows=true
com.knime.server.executor.skip_teamspace_mount=
com.knime.server.executor.start_port=
com.knime.server.executor.sudo_cmd=
com.knime.server.executor.update_metanodelinks_on_load=

It seems KNIME Server cannot find it or something.

ana_ved · January 4, 2021, 8:45am

Hi Rahul,

Did you have a look at our upgrade documentation? For the update to KNIME Executor 4.3, you need to download a fresh new executor.

Best wishes,
Ana

rahul_kumar · January 4, 2021, 9:16am

Hi Ana,

I have followed the documentation and KNIME executor is 4.3 with all extensions.

When I tried to use the message queue and start the executor using startexecutor.sh i get the below error.

!MESSAGE Application error
!STACK 1
java.io.IOException: Could not create a connection to message queue: Connection refused (Connection refused)
at com.knime.enterprise.executor.msgq.RabbitMQHandler.createConnection(RabbitMQHandler.java:194)
at com.knime.enterprise.executor.msgq.RabbitMQHandler.(RabbitMQHandler.java:139)
at com.knime.enterprise.executor.KNIMEExecutorApplication.start(KNIMEExecutorApplication.java:59)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:203)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:137)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:107)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:401)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:255)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:657)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:594)
at org.eclipse.equinox.launcher.Main.run(Main.java:1447)
at org.eclipse.equinox.launcher.Main.main(Main.java:1420)
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:607)
at com.rabbitmq.client.impl.SocketFrameHandlerFactory.create(SocketFrameHandlerFactory.java:60)
at com.rabbitmq.client.impl.recovery.RecoveryAwareAMQConnectionFactory.newConnection(RecoveryAwareAMQConnectionFactory.java:62)
at com.rabbitmq.client.impl.recovery.AutorecoveringConnection.init(AutorecoveringConnection.java:156)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1104)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1063)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:885)
at com.knime.enterprise.executor.msgq.RabbitMQHandler.createConnection(RabbitMQHandler.java:175)
… 15 more

!ENTRY org.eclipse.equinox.preferences 2 2 2021-01-04 09:13:18.802
!MESSAGE Failed to delete preferences file: /home/knime/knime-workspace/.metadata/.plugins/org.eclipse.core.runtime/.settings/org.knime.product.prefs.

!ENTRY org.eclipse.equinox.preferences 2 2 2021-01-04 09:13:18.828
!MESSAGE Failed to delete preferences file: /home/knime/knime-workspace/.metadata/.plugins/org.eclipse.core.runtime/.settings/org.knime.workbench.explorer.view.prefs.

!ENTRY org.eclipse.equinox.preferences 2 2 2021-01-04 09:13:18.830
!MESSAGE Failed to delete preferences file: /home/knime/knime-workspace/.metadata/.plugins/org.eclipse.core.runtime/.settings/org.knime.workbench.core.prefs.

!ENTRY org.eclipse.equinox.preferences 2 2 2021-01-04 09:13:18.885
!MESSAGE Failed to delete preferences file: /home/knime/knime-workspace/.metadata/.plugins/org.eclipse.core.runtime/.settings/org.eclipse.ui.workbench.prefs.

ana_ved · January 4, 2021, 11:22am

Hi @rahul_kumar,

I think I misunderstood your message – I thought you had updated the Executor you already had, but if you downloaded one from the scratch as the upgrade meant this should be fine.

I am just rewriting with a more complete answer: with KNIME Server 4.11, we change the communication method between server and executor from RMI and have officially moved to using qpid.

If you would like to use qpid, you need to add the following line to the KNIME.ini of the executor

-Dcom.knime.enterprise.executor.msgq=amqp://knime:20knime16@localhost/

Additionally, in the knime-server.config, you need to comment out the com.knime.server.executor.knime_exe configuration (just add a # at the start of the line), and check if the following line exists in your server configuration: com.knime.enterprise.executor.embedded-broker=true (if it does not exist, please add it to it).

This way, the embedded message broker will be able to manage the messages between the executor and server correctly

Cheers
Ana

rahul_kumar · January 4, 2021, 1:10pm

Hi Ana,

I already did that but when i start the executer i get the error. On the other note can i still use the RMI executor

WARN main RabbitMQHandler Couldn’t connect to message queue, retry to connect in 10000ms
java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:607)
at com.rabbitmq.client.impl.SocketFrameHandlerFactory.create(SocketFrameHandlerFactory.java:60)
at com.rabbitmq.client.impl.recovery.RecoveryAwareAMQConnectionFactory.newConnection(RecoveryAwareAMQConnectionFactory.java:62)
at com.rabbitmq.client.impl.recovery.AutorecoveringConnection.init(AutorecoveringConnection.java:156)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1104)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1063)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:885)
at com.knime.enterprise.executor.msgq.RabbitMQHandler.createConnection(RabbitMQHandler.java:175)
at com.knime.enterprise.executor.msgq.RabbitMQHandler.(RabbitMQHandler.java:139)
at com.knime.enterprise.executor.KNIMEExecutorApplication.start(KNIMEExecutorApplication.java:59)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:203)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:137)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:107)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:401)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:255)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:657)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:594)
at org.eclipse.equinox.launcher.Main.run(Main.java:1447)
at org.eclipse.equinox.launcher.Main.main(Main.java:1420)

armingrudd · January 4, 2021, 1:19pm

Hi @rahul_kumar,

We had the same issue and we could fix it. We are not sure how exactly it got fixed but here is what I can share with you:
The main issue is:

AMQP Connection 127.0.0.1:5672 :  : RabbitMQHandler :  :  : Couldn't connect to message queue, retry to connect in 10000ms
java.net.ConnectException: Connection refused (Connection refused)

Which makes sense as that’s how the server and executor are communicating:

What we did was check to see if anything was listening on the default port rabbitmq uses (5672)
Then checked to see what process is running.

If there wasn’t anything listening on 5672 that would be the first problem (rabbitmq not starting)

And if there was something like in our case try and make sure it was what we expect.

Our guess is when there was an error, rabbitmq wasn’t running and there may be a race condition of the executor starting too quick.

rahul_kumar · January 4, 2021, 1:40pm

Hi Armin,

We are not using the RAbbitMQ instead we are using the embedded queue using the below option

com.knime.enterprise.executor.embedded-broker=true

rahul_kumar · January 4, 2021, 1:53pm

ana_ved · January 4, 2021, 2:27pm

Hi Rahul,

I will contact you via email.

Best wishes,
Ana

rahul_kumar · January 4, 2021, 3:22pm

Thanks Ana…Please do reply me on mail as i need this to be fixed asap

ana_ved · January 4, 2021, 3:39pm

Hi Rahul,

I have already sent you the email using the email you have provided for your forum account. I have sent you an email again, please let me know if you have received it by replying to it.

Best wishes,
Ana

Dariush · December 30, 2021, 6:40pm

Ana, May we have the answer to this on the forum ? thanx

Iris · December 30, 2021, 6:54pm

@Dariush given this topic is quite old and the information might be outdated, please start a new one describing your problem.

With all KNIME Servers subscriptions you also have access to our Enterprise Support, to create a ticket with us please send an EMail to support@knime.com

Iris · January 1, 2022, 7:00am

This topic was automatically closed after 36 hours. New replies are no longer allowed.