Remove hidden characters

Hello,

I have hidden characters in the data set I got and some special characters.
I was able to remove the carriage returns with the string replacer node but I was not able to remove other hidden characters like the one after the string Pa.m3.mol-1 in my attached example.

Please let me know if there is way to remove my hidden character and/or how to remove hidden characters at all.

Thank you for your help.

Claude.

Example_with_hidden_characters.xlsx (8.5 KB)

Hi @claudeostermann , in reality, there is no such thing as hidden characters. They’re simply “special” characters, in that, regular viewers/editors cannot or don’t know how to display them.

A Hex viewer will always display everything that’s there.

That being said, it’s what I’d suggest to do:

  1. Convert your string to Hex values
  2. Identify and remove the Hex values of these special characters
  3. Convert back to string
1 Like

you could try a regex
sth like


br

1 Like

Hi @claudeostermann , just to demonstrate what I was explaining.

First, though, your Excel file does not really contains “hidden” characters. I think that’s probably because Excel converts them.

This is what I read from your Excel file:
image

And to check if there are any hidden characters, I do a quick join of “XXX” before and after the values in result, and I get:
image

Row3 and Row7 look to have a “hidden” character at the end.

Converting to hex:
image

We can see the hex value “20” at the end of Row3 and Row7. 0x20 in hex is basically a space - I used to build drivers 20 years ago and dealt with these hex values a lot, so these hex values bring back some memories.

So, it looks like Excel is converting them.

Nevertheless, I can still do a demo with the data that you have. As a test, I will remove the degree character “°”, so 69.0 °C for example will become 69.0 C.

The hex code for the degree character is 0xb0, which is in the first 2 records:
image

So, I just need to remove the “b0”:
image

And convert back:
image

As you can see, the degree character has been removed.

You can remove any “hidden” character that you like with this method.

Here’s what the workflow looks like:
image

And here’s the workflow: Remove any special characters.knwf (19.5 KB)

3 Likes

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.