Hi,
I’m running in a strange situation with xPath since upgrading to KNIME 4.4.0. I receive data from an API in JSON format, then convert it to to XML using “JSON to XML” node and then map using xPath. The problem I face is that When I try to do the mapping inside a MetaNode I get the error
attached is a screen grab of error: “The dialog cannot be opened for the following reason: Could not create XML hierarchy tree: The prefix “ns” for attribute “ns:originalKey” associated with an element type “x48” is not bound”. Outside the metanode I can configure the xPath node but not inside. I have asked a college of mine and he cannot even open the configuration outside of the metanode.
I’ve also attached a workflow that reads a copy of the JSON and then convert to XML and then xPath node that errors.
ErrorTest.knar (68.3 KB)
Any help would be appreciated. We did not have this issue in the previous version.
Hi @Willem , while this is an issue that still needs to be looked into, can I ask why do you need to use xpath in your case? I looked into your workflow, and the xpath is trying to get to “/*”, which essentially means get everything, which will give you whatever the previous node (JSON To XML) gives you.
Do you need to use the XPath node?
Hi Bruno,
I haven’t been able to configure the xPath node. I receive the error message when I try to configure the node and that is the issue I’m having.
Hi @Willem , I see, so this error is at the configuration.
I was able to open the configuration on my side (hence how I saw “/*”, which I believe is the default setting), but I’m on v4.3.4.
Must be something in 4.4.0. I have narrowed it down to specific data that is causing the problem. The data for avatarURL which contains pixel sizes are transformed into:
<avatarUrls>
<x48 ns:originalKey="48x48">https://secure.gravatar.com/avatar/2ba676b784afb92bc73637e6dafd52fe?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FPH-3.png</x48>
<x24 ns:originalKey="24x24">https://secure.gravatar.com/avatar/2ba676b784afb92bc73637e6dafd52fe?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FPH-3.png</x24>
<x16 ns:originalKey="16x16">https://secure.gravatar.com/avatar/2ba676b784afb92bc73637e6dafd52fe?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FPH-3.png</x16>
<x32 ns:originalKey="32x32">https://secure.gravatar.com/avatar/2ba676b784afb92bc73637e6dafd52fe?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FPH-3.png</x32>
</avatarUrls>
It is the “ns:originalKey” that is causing the issues. Because it is data I don’t need I was thinking of using the JSON Transformer to remove the data, but I cannot get the format correct so that it would remove the avatarUrls array from all records. in the JSON data.
Or if it is the xPath node that is the issue I’m wondering if there is a way to force a specific version of a node to be used?
I have found that the data that is being converted from JSON:
"avatarUrls" : {
"48x48" : "https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png",
"24x24" : "https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png",
"16x16" : "https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png",
"32x32" : "https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png"
},
into XML of format:
<avatarUrls>
<x48 ns:originalKey="48x48">https://team-1584511028571.atlassian.net/secure/projectavatar?pid=10003&avatarId=10516</x48>
<x24 ns:originalKey="24x24">https://team-1584511028571.atlassian.net/secure/projectavatar?size=small&s=small&pid=10003&avatarId=10516</x24>
<x16 ns:originalKey="16x16">https://team-1584511028571.atlassian.net/secure/projectavatar?size=xsmall&s=xsmall&pid=10003&avatarId=10516</x16>
<x32 ns:originalKey="32x32">https://team-1584511028571.atlassian.net/secure/projectavatar?size=medium&s=medium&pid=10003&avatarId=10516</x32>
</avatarUrls>
Is defiantly causing my issues. When I remove all data containing these types of data then I can configure the XPath node without any issues.
I hope someone from KNIME can have a look into this to confirm if this is a bug in 4.4.0 or am I missing something?
The attached workflow shows this issue, but has to be run in KNIME 4.4.0.
JSONError.knwf (18.4 KB)
Hi @Willem , thank you for the additional info.
I could not open your JSON Reader as I am still on Knime v4.3.4. However, I was able to reproduce what you showed, meaning that this is not something that was introduced in Knime v4.4.x.
I am assuming that the “48x48”, “24x24”, etc are dimensions for the images. It looks like, somehow, Knime is recognizing these as namespaces.
I get the same result as you when converting JSON to XML, it converts these to ns.
Input:
"avatarUrls" : {
"48x48" : "https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png",
"24x24" : "https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png",
"16x16" : "https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png",
"32x32" : "https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png"
},
Expected output:
<avatarUrls>
<48x48>https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png</48x48>
<24x24>https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png</24x24>
<16x16>https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png</16x16>
<32x32>https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png</32x32>
</avatarUrls>
But getting:
<avatarUrls>
<x48 ns:originalKey="48x48" xmlns:ns="http://www.knime.org/json2xml/originalKey/">https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png</x48>
<x24 ns:originalKey="24x24" xmlns:ns="http://www.knime.org/json2xml/originalKey/">https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png</x24>
<x16 ns:originalKey="16x16" xmlns:ns="http://www.knime.org/json2xml/originalKey/">https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png</x16>
<x32 ns:originalKey="32x32" xmlns:ns="http://www.knime.org/json2xml/originalKey/">https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png</x32>
</avatarUrls>
This looks to be by design though. Reading the documentation on JSON to XML:
Node description for JSON To XML
Every non-letter or number or _ character will be removed from the keys, which might cause ambiguity or create empty names -but will be kept as an attribute with key: ns:originalKey-, so it is recommended to stick to English letters in JSON object keys.
I am not sure why the node is designed to behave like this. Other converters outside Knime are able to convert to the expected results. Tagging @ScottF and @Iris from the Knime Team to get more info about this rule.
I am not sure what your options would be @Willem , apart from manipulating the JSON string. I tested with a few variations to see what would work:
Modified input:
{
"avatarUrls" : {
"d48x48" : "https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png",
"24x24d" : "https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png",
"16_x_16" : "https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png",
"32x32" : "https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png"
}
}
Output:
<avatarUrls>
<d48x48>https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png</d48x48>
<x24d ns:originalKey="24x24d" xmlns:ns="http://www.knime.org/json2xml/originalKey/">https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png</x24d>
<_x_16 ns:originalKey="16_x_16" xmlns:ns="http://www.knime.org/json2xml/originalKey/">https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png</_x_16>
<x32 ns:originalKey="32x32" xmlns:ns="http://www.knime.org/json2xml/originalKey/">https://secure.gravatar.com/avatar/24a92b5052d8ca700e92f0bc390bdeff?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAG-2.png</x32>
</avatarUrls>
So, it looks like if it starts with a character (the d48x48), it is detected as string and gets translated as expected.
Thank you @bruno29a for your valuable feedback. I suspect then it must be that the API I’m using has been changed to include this new data but I will have to confirm with JIRA as my workflow worked last I worked on it, about a month or two ago.
I have managed to get my workflow to work again by using the JSON Transform node to remove those specific items that are causing the issues.
One good thing came of this, I had to learn how JSON Path node works, I used to just convert to XML and use XPath as I was used to it.
Would be interesting to know why this is happening and if there is a better way of handling it though, so waiting eagerly for info from @ScottF and @Iris .
Hello @Willem,
tnx for providing workflow example and doing investigation on your own (and to @bruno29a as well). I have looked at it and indeed something’s seems wrong so opened up a ticket for developers to look more into it. Upon news someone will update this topic.
Br,
Ivan
Hi @ipazin , thanks for looking into this and for opening a ticket. The node is definitely not giving the expected equivalent XML version of the JSON, however, as I mentioned, it seems like for this type of data, it looks like it’s by design for some reason, based on the node’s documentation.
So, the node is behaving as what’s in the documentation, but it’s not giving the expected results. I don’t see why it has these restrictions, perhaps it was trying to warn against “special” characters such as accented characters, or characters from non-English languages as it would have trouble working with these characters, but it should be able to properly understand numbers (0-9) and underscores.
It seems that it could be naming restrictions in XML that is actually the problem. It seems like names should not start with a number. This may be the issue, I am going to contact the cloud system’s support and find out if they have added the extra data recently as that is what I think is what happened since it was all working a month or so ago when I originally created the workflow. It may just be a restriction with XML and I will have to modify my workflow to handle this situation accordingly.
But it would still be good to have confirmation if this could be fix or if all is working as expected.
Hi @Willem , it should not be a naming restrictions. As I mentioned, other converters are able to convert your sample JSON data to XML properly.
I’ve noticed that the workflow I previously included may not have the included sample data. Here is a new copy of the workflow.
JSONError.knwf (41.4 KB)
Hi @bruno29a, I’ve done a conversion online and then copied the result into Notepad++ and the naming seems to be the issue, attached file is screen grab of error in NotePad++
Hello @Willem,
in case you haven’t seen or tried it out, with KNIME version 4.4.1, this issue shouldn’t happen anymore.
Find the changelog here: KNIME Analytics Platform 4.4 | KNIME
Br,
Ivan
Hi @ipazin , thanks for the update, but there was nothing to “fix” in the end.
I’ve finally moved to Knime v4.4.1 (from v4.3.4) yesterday night. Just to confirm that the behaviour still happens, but it is expected.
Just like @Willem pointed out, <48x48> (or <24x24>, or <16x16>, etc) is not a valid XML format. While some converters still converted to <48x48>, XML validators flag this as an invalid format. That is why it’s converted to <x48 ns:originalKey="48x48">.
In the end, it’s more of a data issue.
Hello @bruno29a,
if I got it right issue was with JSON to XML node creating invalid XML leading to Dialog cannot be opened message/error from XPath node. With version 4.4.1 XPath node’s configuration dialog can be successfully opened in above workflow example.
Br,
Ivan
Hi @ipazin , indeed, that was the original issue 
Unfortunately, I never reproduced the original issue, and now I just moved to 4.4.1, so I can’t compare.
Thanks for clarifying 
A post was split to a new topic: Offline activities