Issue with JSONPath character encoding

[peterdstraw]

Hello,

First time posting so I hope this is the correct place.

I have an issue with JSONPath where certain characters are being converted to their unicodes when the node is executed.

For instance, with this JSON:

{“items” : ["‘sdfsf’"]}

The JSON path node converts the left and right single quote characters to unicodes and outputs this:

["\u2018sdfsf\u2019"]

I’ve tried changing the character encoding both in the preferences and in the configuration file but this hasn’t changed the behaviour.

Is there a way to avoid this behaviour?

Thanks.

[ipazin]

Hello @peterdstraw,

and welcome to KNIME Community! No worries, this is the right place for KNIME related questions. (And occasionally non KNIME related questions which get addressed as well )

Tried your example and seeing no conversion to Unicode. What OS and what KNIME version are you using? Can you share workflow example with us where this can be seen? Simply add example JSON to Table Creator node followed by configured JSON Path node. Upload here or on KNIME Hub.

Br,
Ivan

[peterdstraw]

Hi @ipazin, thanks for the reply.

Here’s a simple example. I’ve tried this on both 4.1.2 and 4.3.0 and get the same behaviour.

I’m running on Windows 10 and have the issue on two different machines.

JSON example.knwf (6.5 KB)

[peterdstraw]

I’ve worked around the issue by converting the JSON cell to a string and using Regex to find the relevant parts of the string for my project. Converting to a string in string manipulation doesn’t cause the same behaviour and the characters are preserved.

[ipazin]

Hello @peterdstraw,

glad you found a way. Seems those are curly single quotes that are not handled well. Another workaround is to replace them (or remove them if not needed) with straight single quotes and then JSON Path node works as expected. For this I used following expressions in String Manipulation node:

replaceChars($JSON Column$,"‘’" ,"''" )

Br,
Ivan

[peterdstraw]

Thanks for looking at this @ipazin. It’s more than just those characters which are affected unfortunately. We’ve had similar issues with accented letters, etc. It makes the JSON Path node more or less unusable for my purposes as far as I can see but as you say there are some workarounds so I’ve been able to get to what I needed.

[ipazin]

Hello @peterdstraw,

you are welcome. I see. Seems like there might be some room for improvement. Will make sure to forward it.

Br,
Ivan