Delete rows with special characters

[jpages]

Hello,

I recently had a problem that I solved manualy with a row filter node but I want to know if it is possible to do it automaticly with another method.

So I have a .csv with alot of rows and in some of them I find character like:

Or like:

I need a method that eliminates the rows with this kind of registers, I tryied with manipulation string search and replace funtion but it doesn’t detect this kind of character so it doesn’t work.

Any ideas?

Thanks!

[aworker]

Hi @jpages

I have a question concerning your problem. Do you know what is the set of special characters which should make a row to be removed ?

Best

Ael

[bruno29a]

Hi @jpages , the search will always find what you are asking it to find. Chances are that you are not searching for the correct characters.

I’m not sure how you determine which characters to search. What you are seeing is simply gibberish, caused by your system not being able to properly display the characters, most certainly caused by your current character set not supporting these characters. So you can’t just search for what you are seeing.

There are a few ways to handle this or to capture the correct characters.

One way that does not fail for me is to read the text in hex. Hex will always return the proper hex value of the special characters, and you can remove them properly that way.

EDIT: You can check this thread where I posted a demo to convert to hex and remove some characters: Remove hidden characters

Here’s another thread that can help: Remove special characters from the text - #6 by bruno29a

[jpages]

No, just this kind of characters I marked as yellow. The point of this is that when I try to import this data on postgres I’m having enconding errors, so I need to remove this rows.

Thanks.

[Daniel_Weikert]

Do you have a sample?
If you like to delete the rows the using row filter as you do only requires one node? (You could use maybe have used regex)
I don’t see a way more efficient or am I missing something?
br

[jpages]

I used 2 nodes one for each row I needed to eliminate.