REST read response node fails for (some) valid JSON return values

Hi KNIMErs,

I was playing around with the following REST resource in KNIME using a Get Resource and a Read REST Response node : 

https://www.ebi.ac.uk/unichem/rest/cpd_search/CHEMBL490/1/9/0/4/0/0/0/0/0

I get it to work but as you can see the response is not really 'flat', and in order to flatten it I need a second Read REST Response node to flatten it sufficiently, which is not really nice.

The 1/9/0/4/0/0/0/0/0 bit of the query are some flags .. turns out that setting the last flag to 1:

https://www.ebi.ac.uk/unichem/rest/cpd_search/CHEMBL490/1/9/0/4/0/0/0/0/1

returns a flatter data structure which I should be able to parse more easily.

However for some reason inexplicable to me the Read Response node throws a null pointer exception .. from the JSON point of view everything looks valid, i.e. the service returns valid JSON syntax.

I can provide an example workflow, or the two different response bodies if that would be helpful in order to see what's going (wr)on(g).

 

Thanks!!

 

Benjamin

Hi Benjamin,

I have checked zour representations.

The NullPointerException in the second one (with the 1 flag) is caused by a null value in the JSON representation. The JSON converter did not expect something like this, yet. I will adress this in the next version. 

However a fast fix is to put a StringManipulator Node between the GET Representation and the Read REST Representation node. Select "Replace Column" and add something like "replace($Representation$, "null", "\"null\"")" to the Expression field. This will replace the null values by a "null" string.

 

As for the first representation (without the 1 flag), it is very hard to automatically convert an unknown representation into a table, especially if there is some sort of metadata flanking the actual table data. I would not know how to distinguish these two. Instead of guessing (which might lead to loss of data) we decided to include everything by default and leave it user to get rid of unrequired parts.

At the moment we have an option in the Read Representation node which allows you to specify the names of JSON objects that shall be ignored by the parser. To convert the first representation in one step switch the data format to JSON (instead of Autodetection), this will reveal the parsing options. Then type the names of the objects to be ignored in the "Ignore object called:" field and press "Add". In your case you have to ignore the following objects: 1, name, description, name_long, base_id_url. This will produce the same table like the second URL.

Hope, I could help you.

Anna

 

 

 

 

 

Hi Anna,

thanks a lot -- that works great!!

cheers

Benjamin