Remove hidden characters

[claudeostermann]

Hello,

I have hidden characters in the data set I got and some special characters.
I was able to remove the carriage returns with the string replacer node but I was not able to remove other hidden characters like the one after the string Pa.m3.mol-1 in my attached example.

Please let me know if there is way to remove my hidden character and/or how to remove hidden characters at all.

Thank you for your help.

Claude.

Example_with_hidden_characters.xlsx (8.5 KB)

[bruno29a]

Hi @claudeostermann , in reality, there is no such thing as hidden characters. They’re simply “special” characters, in that, regular viewers/editors cannot or don’t know how to display them.

A Hex viewer will always display everything that’s there.

That being said, it’s what I’d suggest to do:

1. Convert your string to Hex values
2. Identify and remove the Hex values of these special characters
3. Convert back to string

[Daniel_Weikert]

you could try a regex
sth like

br

[bruno29a]

Hi @claudeostermann , just to demonstrate what I was explaining.

First, though, your Excel file does not really contains “hidden” characters. I think that’s probably because Excel converts them.

This is what I read from your Excel file:

And to check if there are any hidden characters, I do a quick join of “XXX” before and after the values in result, and I get:

Row3 and Row7 look to have a “hidden” character at the end.

Converting to hex:

We can see the hex value “20” at the end of Row3 and Row7. 0x20 in hex is basically a space - I used to build drivers 20 years ago and dealt with these hex values a lot, so these hex values bring back some memories.

So, it looks like Excel is converting them.

Nevertheless, I can still do a demo with the data that you have. As a test, I will remove the degree character “°”, so 69.0 °C for example will become 69.0 C.

The hex code for the degree character is 0xb0, which is in the first 2 records:

So, I just need to remove the “b0”:

And convert back:

As you can see, the degree character has been removed.

You can remove any “hidden” character that you like with this method.

Here’s what the workflow looks like:

And here’s the workflow: Remove any special characters.knwf (19.5 KB)