NullPointerException in POST Resource

All,

I stumbled on the follwoing problem with the KREST nodes. Whenever I do a succesful post, the node returns a NullPointerException.

Here's what is returned from the HTTPS call:

HTTP/1.1 200 OK
Cache-Control: no-cache
Pragma: no-cache
Expires: -1
Server: Microsoft-IIS/8.0
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Tue, 14 Apr 2015 09:09:05 GMT
Content-Length: 0

Here's what is in the knime.log:

2015-04-14 11:03:01,389 DEBUG main POST Resource : reset
2015-04-14 11:03:01,389 WARN  main POST Resource : Please regard that reseting the node will not undo requests you have sent to the server!
2015-04-14 11:03:01,389 DEBUG main POST Resource : clean output ports.
2015-04-14 11:03:01,390 DEBUG main NodeContainer : POST Resource 0:101:1 has new state: IDLE
2015-04-14 11:03:01,392 DEBUG main POST Resource : Configure succeeded. (POST Resource)
2015-04-14 11:03:01,392 DEBUG main NodeContainer : POST Resource 0:101:1 has new state: CONFIGURED
2015-04-14 11:03:03,994 DEBUG main ExecuteAction : Creating execution job for 1 node(s)...
2015-04-14 11:03:03,994 DEBUG main NodeContainer : POST Resource 0:101:1 has new state: CONFIGURED_MARKEDFOREXEC
2015-04-14 11:03:03,994 DEBUG main NodeContainer : POST Resource 0:101:1 has new state: CONFIGURED_QUEUED
2015-04-14 11:03:03,994 DEBUG main NodeContainer : Post Predictions 0:101 has new state: EXECUTING
2015-04-14 11:03:03,994 DEBUG main NodeContainer : AppDet_Predict 0 has new state: EXECUTING
2015-04-14 11:03:03,995 DEBUG KNIME-Worker-47 WorkflowManager : POST Resource 0:101:1 doBeforePreExecution
2015-04-14 11:03:03,995 DEBUG KNIME-WFM-Parent-Notifier NodeContainer : ROOT  has new state: EXECUTING
2015-04-14 11:03:03,995 DEBUG KNIME-Worker-47 NodeContainer : POST Resource 0:101:1 has new state: PREEXECUTE
2015-04-14 11:03:03,995 DEBUG KNIME-Worker-47 WorkflowManager : POST Resource 0:101:1 doBeforeExecution
2015-04-14 11:03:03,995 DEBUG KNIME-Worker-47 NodeContainer : POST Resource 0:101:1 has new state: EXECUTING
2015-04-14 11:03:03,995 DEBUG KNIME-Worker-47 WorkflowFileStoreHandlerRepository : Adding handler a3dd3d92-0997-452a-b100-3af92ad4a3a3 (POST Resource 0:101:1: <no directory>) - 1 in total
2015-04-14 11:03:03,995 DEBUG KNIME-Worker-47 LocalNodeExecutionJob : POST Resource 0:101:1 Start execute
2015-04-14 11:03:03,995 INFO  KNIME-Worker-47 SingleRestNodeModel : Execute REST Node ...
2015-04-14 11:03:04,307 DEBUG KNIME-Worker-47 POST Resource : reset
2015-04-14 11:03:04,307 WARN  KNIME-Worker-47 POST Resource : Please regard that reseting the node will not undo requests you have sent to the server!
2015-04-14 11:03:04,307 ERROR KNIME-Worker-47 POST Resource : Execute failed: ("NullPointerException"): null
2015-04-14 11:03:04,307 DEBUG KNIME-Worker-47 POST Resource : Execute failed: ("NullPointerException"): null
java.lang.NullPointerException
	at com.cenix.krest.nodes.submitter.single.SingleResponseHandler.getDataFormatForResponse(SingleResponseHandler.java:312)
	at com.cenix.krest.nodes.submitter.single.SingleResponseHandler.<init>(SingleResponseHandler.java:75)
	at com.cenix.krest.nodes.submitter.single.SingleRestNodeModel.processResponse(SingleRestNodeModel.java:230)
	at com.cenix.krest.nodes.submitter.single.SingleRestNodeModel.execute(SingleRestNodeModel.java:215)
	at org.knime.core.node.NodeModel.execute(NodeModel.java:706)
	at org.knime.core.node.NodeModel.executeModel(NodeModel.java:555)
	at org.knime.core.node.Node.invokeFullyNodeModelExecute(Node.java:1131)
	at org.knime.core.node.Node.execute(Node.java:927)
	at org.knime.core.node.workflow.NativeNodeContainer.performExecuteNode(NativeNodeContainer.java:559)
	at org.knime.core.node.exec.LocalNodeExecutionJob.mainExecute(LocalNodeExecutionJob.java:95)
	at org.knime.core.node.workflow.NodeExecutionJob.internalRun(NodeExecutionJob.java:179)
	at org.knime.core.node.workflow.NodeExecutionJob.run(NodeExecutionJob.java:110)
	at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:328)
	at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:204)
	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:125)
	at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:248)
2015-04-14 11:03:04,308 DEBUG KNIME-Worker-47 WorkflowManager : POST Resource 0:101:1 doBeforePostExecution
2015-04-14 11:03:04,308 DEBUG KNIME-Worker-47 NodeContainer : POST Resource 0:101:1 has new state: POSTEXECUTE
2015-04-14 11:03:04,308 DEBUG KNIME-Worker-47 WorkflowManager : POST Resource 0:101:1 doAfterExecute - failure
2015-04-14 11:03:04,308 DEBUG KNIME-Worker-47 POST Resource : reset
2015-04-14 11:03:04,308 WARN  KNIME-Worker-47 POST Resource : Please regard that reseting the node will not undo requests you have sent to the server!
2015-04-14 11:03:04,308 DEBUG KNIME-Worker-47 POST Resource : clean output ports.
2015-04-14 11:03:04,308 DEBUG KNIME-Worker-47 WorkflowFileStoreHandlerRepository : Removing handler a3dd3d92-0997-452a-b100-3af92ad4a3a3 (POST Resource 0:101:1: <no directory>) - 0 remaining
2015-04-14 11:03:04,308 DEBUG KNIME-Worker-47 NodeContainer : POST Resource 0:101:1 has new state: IDLE
2015-04-14 11:03:04,309 DEBUG KNIME-Worker-47 POST Resource : Configure succeeded. (POST Resource)
2015-04-14 11:03:04,309 DEBUG KNIME-Worker-47 NodeContainer : POST Resource 0:101:1 has new state: CONFIGURED
2015-04-14 11:03:04,309 DEBUG KNIME-Worker-47 NodeContainer : Post Predictions 0:101 has new state: IDLE
2015-04-14 11:03:04,309 DEBUG KNIME-Worker-47 NodeContainer : AppDet_Predict 0 has new state: IDLE
2015-04-14 11:03:04,309 DEBUG KNIME-WFM-Parent-Notifier NodeContainer : ROOT  has new state: IDLE

Doesn't the POST resource node support an empty (meaning Content-Length=0) HTTP 200 response? Or is there something else wrong?

Tim

Additional info:

If I interpret the code correctly, the problem comes from the fact that there is no 'Content-Type' header, because obviously, there is no content. This is the code:

public DataFormat getDataFormatForResponse() {
	    String displayName = response.getType().toString();
	    try {
	        ContentType contentType = ContentType.fromDisplayName(displayName);
	        return contentType.getDataFormat();
        } catch (UnknownContentTypeException ucte) {
            this.unknownContentType = ucte.getContentType();
            return ContentType.guessDataFormat(unknownContentType);
	    }
	}

response.getType returns null, and then it goes wrong.

If I look at the code for KREST2, it looks slightly different:

	public DataFormat getDataFormatForResponse() {
	    MediaType mediaType = response.getMediaType();
	    String displayName = null;
	    if (mediaType != null) {
	        displayName = response.getMediaType().toString();
	    }
	    try {
	        ContentType contentType = ContentType.fromDisplayName(displayName);
	        return contentType.getDataFormat();
        } catch (UnknownContentTypeException ucte) {
            this.unknownContentType = ucte.getContentType();
            return ContentType.guessDataFormat(unknownContentType);
	    }
	}

This code is different: content-type is fetched from the java library (instead of jersey) and it is checked first.

So KREST2 might behave better, but I have no clue on how to install it.
As this is a different issue, I thought this was worth a separate post : http://tech.knime.org/node/47917/view

Tim

I have changed the KREST (not KREST2) trunk code (seems I have write access other community projects too), so the next nightly build might work well. In case you know a public service that responds empty results, I would appreciate it for testing. (It would also help if those who had problems could test whether it was fixed for them or not.)

Thank you for the detailed analysis. --gabor

Still no new build for KREST:

  • KNIME Community Contributions - Others -> REST Client shows current ver. 1.2.7.201502191505
  • KNIME Community Contributions (nightly build) - Others -> REST Client shows much older ver. 1.2.7.201402251452

To get nightly builds, I added http://update.knime.org/community-contributions/trunk to the Available Software Sites in preferences. Any other way to get an updated plugin?

I apologise for the problem. It seems the krest2 nodes in the build made impossible to create the original too (just got access to the KREST jenkins job). I'll try to fix that, though have not seen errors like that before. The good news: probably both will work once it get fixed.

Edit: Changed the parts causing build problems. A custom build was already successful, the tests were successfully executed. The next nightly build will hopefully include the fix (and maybe also KREST2?) in the nightly update site. In case not, I'll share the jenkins-built update site somewhere so you can try out there and if it works for you, I'll copy the bugfix to the stable branch (assuming I have permission to do that).

Thanks for testing.

I see new build on nightly - however, this is KREST2 (com.cenix.krest2.nodes.feature.feature.group) ver. 2.0.0.201505281334.

I must admit that having both versions of KREST is quite confusing as the nodes have exactly the same names and appear duplicated in the same folder.

Not sure why but my NullPointerException problem still appears after using new KREST2 nodes - both GET and POST commands.

Will wait until tomorrow to see new stuff being available for update.

Yeah, it is confusing, though the old one is still available with the 1.2.7.2015... version number. The fix should just work with the nodes created with the old node. I am not sure how much changes may I do in the trunk version -as it seems there will be a maintainer (in a foreseeable future) and I do not want to cause more harm than helping-, so I will probably not remove the KREST2 nodes/preferences from the build, but maybe that would not hurt if it had a 2 in their category/node names. I'll probably have no time to do that before the end of this week though.

Thank you for checking.
 

Since the current nightly it is easier to distinguish the REST 2 and REST nodes (both in the Node repository and in the preferences) when both are installed. (Though the changes are not backward compatible and the KREST2 nodes might change drastically.)

Just installed the latest update - indeed I see KREST and KREST2 nodes separated in 2 folders under Community Nodes. However, there is no way to distinguish between nodes with the same name - for example, GET Resource looks the same in both folders. It would be much better if some suffix was added to identify the version.

BTW, is there a way to use less popular or customer headers in requests? See separate discussion.

Actually, there is a way. If you scroll down to the bottom of the node decription, you will see a 2 in the grey text (This node is contained in REST Client 2 Node extension for KNIME Workbench provided by AABerger.) for the KREST 2 nodes. If there is a need to make it more visible, I might change the node's name, just wanted to make things less invasive as most probably people will not install both KREST 1 and KREST 2 for their KNIME installation.

I do not want to interfere with the developments of the future maintainers of KREST 2 (I was informed by KNIME developers that a team -which is already community contributor- is considering to take the development for the KREST nodes), so I would prefer not to implement the custom headers change for KREST 2 (or 1). My contribution might not be up to their standards or they might prefer a different solution, so I think this is better this way.

As for custom headers, I am afraid the only workaround is currently KNIME is use a different tool, like Java Snippets (probably with apache commons httpclient?), R (RCurl?) or Python (urllib2/3).

I hope the POST service call with 0 length result is working fine though.

BTW, is there a way to use less popular or customer headers in requests? See separate discussion.

fyi: Newer versions of the HttpRetriever in the Palladian nodes support arbitrary HTTP headers.