Feature request: External Tool/External Tool (Labs) - select encoding for Input and Output files

Dear community. Need some help.

In my workflow i want to do some processing with standalone pyhton script, so using External Tool or External Tool (Labs) nodes. The problem is that my data contains Cyrillic characters. And i want to get them back in correct encoding with Output file from python script. Unfortunately, is that there is no possibility to select in the node configuration in what encoding you want to process data for Input and Output files (like in read/write csv files, for example). By default it is done in encoding that do not support Cyrillic, and if you use in python script “encoding=‘utf8’” statement for reading file then you get an error:

UnicodeDecodeError: ‘utf-8’ codec can’t decode byte 0xcc in position 89: invalid continuation byte.

Is it possible to add such functionality to mentioned nodes?

Maybe you also know some workaround to overcome the issue.

Thanks in advance,
Alex.

The CSV Writer, at least, has an encoding configuration pane (SS attached) - which reader and writer nodes are you specifically trying to use?

Hi.

As it mentioned in the topic, I’m referring to External Tool node and External Tool (Labs) node.

And it will be nice to have configuration pane you showed on the screen in those nodes.

BR,
Alex.

Hi - Sorry to be obtuse about this, but i meant this in the context of doing something like this:

In this fashion, you can use writer and reader nodes with flexible encoding, and still have your external tool do whatever.

2 Likes

Thanks for the suggestion.

I will pay around with it.

BR,
Alex.

Dear @quaeler.

Proposed solution is working. And I managed to integrate it into my global workflow. Thanks once again!

Few things to bare in mind. In the External Tool you have to specify fake Input and Output files. Otherwise the input file will be overwritten without UTF-8 encoding and then output file will be incorrectly read. Correct files should be used in the python script itself.

Anyway. This is only a workaround that introduce some additional complexity in the workflow… If there is a way to update External Tool node and implement encoding selection functionality in it, that will be really nice!

BR,
Alex

1 Like

Glad to hear that it’s a helpful hack - i agree it is completely awkward and not a real solution; i’ve opened up a ticket for your feature request.

2 Likes

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.