KNIME Server fails with NullPointerException

Hello

We have a job which is executed in everyday’s 8:30 am, It works well ago.
But the job failed with NullPointerException the day before yesterday.
yesterday and today, the error do not happen.

Error Log is the following:

12-Sep-2020 08:33:25.363 ERROR [http-nio-8080-exec-10] com.knime.enterprise.server.rest.impl.providers.exception.ExecutorExceptionMapper.toResponse NullPointerException
com.knime.enterprise.utility.ExecutorException: NullPointerException
at com.knime.enterprise.executor.rmi.RMIUtil.checkAndLog(RMIUtil.java:128)
at com.knime.enterprise.executor.rmi.RMIUtil.runWithContextFinder(RMIUtil.java:94)
at com.knime.enterprise.executor.rmi.RMIWorkflowExecutor.loadWorkflow(RMIWorkflowExecutor.java:458)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
at sun.rmi.transport.Transport$1.run(Transport.java:200)
at sun.rmi.transport.Transport$1.run(Transport.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:283)
at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:260)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:227)
at java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:179)
at com.sun.proxy.$Proxy155.loadWorkflow(Unknown Source)
at com.knime.enterprise.server.executor.rmi.RMIExecutorImpl.loadWorkflow(RMIExecutorImpl.java:795)
at com.knime.enterprise.server.jobs.WorkflowJobManagerImpl.loadWorkflow(WorkflowJobManagerImpl.java:699)
at com.knime.enterprise.server.application.jobs.PermissionAwareWorkflowJobManagerImpl.loadWorkflow(PermissionAwareWorkflowJobManagerImpl.java:101)
at com.knime.enterprise.server.rest.impl.v4.repository.JobsImpl.createNewJobAsMason(JobsImpl.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.openejb.server.cxf.rs.PojoInvoker.performInvocation(PojoInvoker.java:43)
at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96)
at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:191)
at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:101)
at org.apache.openejb.server.cxf.rs.AutoJAXRSInvoker.invoke(AutoJAXRSInvoker.java:68)
at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)
at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:267)
at org.apache.openejb.server.cxf.rs.CxfRsHttpListener.doInvoke(CxfRsHttpListener.java:253)
at org.apache.tomee.webservices.CXFJAXRSFilter.doFilter(CXFJAXRSFilter.java:94)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.openejb.server.httpd.EEFilter.doFilter(EEFilter.java:65)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.knime.enterprise.server.rest.NoContentTypeFilter.doFilter(NoContentTypeFilter.java:121)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.knime.enterprise.server.rest.StaticContentFilter.doFilter(StaticContentFilter.java:110)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.knime.enterprise.server.rest.CSRFFilter.doFilter(CSRFFilter.java:103)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.knime.enterprise.server.rest.impl.LicenseCheckFilter.doFilter(LicenseCheckFilter.java:111)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.knime.enterprise.server.rest.ProxyRewriteFilter.doFilter(ProxyRewriteFilter.java:166)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.knime.enterprise.war.GeneralSecurityHeadersFilter.doFilter(GeneralSecurityHeadersFilter.java:79)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.tomee.catalina.OpenEJBValve.invoke(OpenEJBValve.java:44)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:610)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
at org.apache.tomee.catalina.OpenEJBSecurityListener$RequestCapturer.invoke(OpenEJBSecurityListener.java:97)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:650)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:800)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:800)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1471)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)

Seems error happened during sending post to invoke a workflow( [Call Workflow (Table Based)]):

127.0.0.1 - knimeadmin [12/Sep/2020:08:33:24 +0900] “DELETE /knime/rest/v4/jobs/f66caf93-d18d-4e9e-a677-61c74d8b84c0 HTTP/1.1” 204 -
127.0.0.1 - knimeadmin [12/Sep/2020:08:33:25 +0900] “POST /knime/rest/v4/repository/WF_004_LoadData_ByIdentifier:jobs?timeout=60000&wizardExecution=false HTTP/1.1” 500 21
127.0.0.1 - knimeadmin [12/Sep/2020:08:33:26 +0900] “POST /knime/rest/v4/repository/WF_004_LoadData_ByIdentifier:jobs?timeout=60000&wizardExecution=false HTTP/1.1” 201 3960
127.0.0.1 - knimeadmin [12/Sep/2020:08:33:26 +0900] “POST /knime/rest/v4/repository/WF_004_LoadData_ByIdentifier:jobs?timeout=60000&wizardExecution=false HTTP/1.1” 201 3960
127.0.0.1 - knimeadmin [12/Sep/2020:08:33:26 +0900] “POST /knime/rest/v4/jobs/30b1cd1d-07f4-4ce9-81b1-809c6d6b04a9?reset=false&async=true&timeout=600000 HTTP/1.1” 200 3312
127.0.0.1 - knimeadmin [12/Sep/2020:08:33:26 +0900] “POST /knime/rest/v4/jobs/fdfd7e6e-68b0-49be-8aaf-c9b51fe09d5c?reset=false&async=true&timeout=600000 HTTP/1.1” 200 3312

How to solve the problem?
Thanks in advance.
Ryu

Hi @laughsmile,

as you already recognized the error seems to happen during the loading of a workflow. It seems that there were two REST calls sent to execute the same workflow in parallel? Was this intended?

Which server and executor versions are you using?

Best regards,
Michael

Hi @MichaelRespondek

Thank you for your response.
We are Using Parallel Chunk Loop to call workflows by using “Call workflow (table based)” .
Table Creator(6 Rows) -> Parallel Chunk Start-> Call workflow (table based) -> Parallel Chunk End
※ CPU has a 4-core processor, 8 logical processors by the result of msinfo32 command
※ KNIME Using 8G Memory

KNIME Sever : 4.82
KNIME ANALYTICS PLATFORM: 3.72
Java version is 1.8.0_152 , and OS is Windows Server 2016

Thanks in advance
Ryu

Hi Ryu,

we had a bug regarding the Chunk Loop nodes that was fixed in version 4.0.1 of the Analytics Platform but as this was only affecting workflows with large parallelization I don’t think that this is the error you got here.
Also there were some bugfixes regarding the call workflow nodes.

Is it possible for you to update the server to a more recent version?

In parallel I will send you a mail and ask for the logfiles of the server and executor to analyze your issue further.

Best regards,
Michael