File separator

[Thiemo.Kellner]

Hi

I want to construct the file path dynamically in a loop to load data from several files into a database. The workflow looks like this, so far.

I want to do the path construction in the string manipulation node as follows.

string(joinSep("/",
               $${SC_PATH_DIRECTORY_DATA}$$,
               replace(replace($${SC_PATTERN_FILENAME}$$,
                               $${SC_PLACEHOLDER_KANTON}$$,
                               lowerCase($${Sabkuerzung}$$)),
                       $${SC_PLACEHOLDER_YEAR2DIG}$$,
                       $${Sv_year}$$)))

I work on Linux, so my file separator is /, however, I would like to have it platform independent.

To use the “Extract System Properties” node seems awfully awkward to me. Does KNIME provide such properties in a direct way?

Cheers

Thiemo

[bruno29a]

Hi @Thiemo.Kellner , if it’s just a matter of separator, I believe Windows understands the “/” as separator. You can, for example, point to C:\Users\user1\Documents\file.txt or C:/Users/user1/Documents/file.txt in the File Reader, they should both work.

However, the differences between Windows and Linux is more than just the file separator. Windows uses the concept of drive letter (A:, B:, C:, D:, …, Z:) while linux uses drive names (/dev/hda/, etc) or partitions (/var, /usr, etc) or mount points.

[Thiemo.Kellner]

@bruno29a thanks for the quick reply. I noticed that Windows at least sometimes (cmd prompt) handles also / as file separator, though I am not sure whether this is universal regardless of the client. Still, I am well aware of the drive letter/mount point, but the user has to adapt the workflow variables (C_PATH_DIRECTORY_DATA, …) to one’s need. I could solve this by packing the separator into a workflow variable too, but again, it would feel awkward to me as this is an information that should be possible to handle without user interaction.

Be it as it may, I am also curious, in general, whether there is a straight forward way to access platform information.

[bruno29a]

Hi @Thiemo.Kellner , no problem.

As far as Knime is concerned, it seems to understand both / and \ as separator on Windows.

Regarding the platform information, the Extract System Properties should do - I saw your comment about this node. This is a quite useful node, and it does give you a lot of information. If it’s too much info, you can actually select which info you want to retrieve. You just need to uncheck the box “Extrat all available properties”, and then Exclude what you don’t need:

If you want to retrieve the OS name, you can extract the “org.osgi.framework.os.name” or/and “os.name”, and based on this value, you can decide which separator to use.

Personally, I think it’s safe to just use “/” and skip the check on the OS. After building the file path, just make sure that you actually convert that string into a type Path - you can use the String to Path node for that.

[Thiemo.Kellner]

Thanks for detailing.

To me, it seemed, that the “Fixed Width File Reader” would just accept the file path as a string.

[Thiemo.Kellner]

Funny, file location box does work with string, using workflow variables requires URL type. shrugg

[diego_rod_lop]

Hey @Thiemo.Kellner
Thanks for your post. I guess in this case could be useful to use the new path functions enhancement in the Expressions nodes: Colummn Expressions and Variable Expressions.

With this new feature you can build for example a local path using strings as an input, and I think this local path is using by default the system configuration where the workflow is being executed, you can give it a try.

Take a look to the example workflow that shows how to use this kind of nodes in the below link

File Path Expressions Example

Hope this could be useful for your specific case.

ps. you can can also close a Table row to variable loop start node with a normal “Loop end” one (connecting it to your Fixed Width File Reader node)

[Thiemo.Kellner]

Hola @diego_rod_lop

Thanks for your suggestions. After upgrading to 4.6.0 I can feel the magic. Thanks!

I now have the problem, that the path variable is not displayed in the file reader.

[diego_rod_lop]

Nice that you have upgrade your KNIME AP version
Don’t hesitate to give us any feedback about the new 4.6 version here and about the new KNIME modern UI Preview

That being said, regarding your last comment I guess the node is expecting an string variable, could you try to append getPathString() before your code and change the variable type to string? See the below screenshot

Lmk if it works!

[Thiemo.Kellner]

I have come from that direction. It gives me:

WARN Fixed Width File Reader 3:20 Errors loading flow variables into node : Coding issue: Cannot create URL of data file from '/home/thiemo/Documents/software/quellensteuer/daten/tar22ag.txt’in filereader config

[bruno29a]

Hi @Thiemo.Kellner , I’ve never used the Fixed Width File Reader. Any reason why you need to use this node as opposed to, let’s say, the File Reader node?

I’m not sure what the Fixed Width File Reader is expecting as a URL, but the File Reader can read from a Path, and I’m sure is configurable to read fixed width I think.

[Thiemo.Kellner]

I thought it might be more convenient to read a fixed length file with a fixed file reader. That’s all.

[Thiemo.Kellner]

I suppose, I need to figure out how to create a URL from a string as workflow variable - though, I have not seen yet (very little have I seen) a URL type workflow variable.

[diego_rod_lop]

yeah as @bruno29a suggested I guess you can use a File Reader node.
In this case is accepting the variable as a path, so no need to convert it into string

Also if you need to use the Fixed Width File Reader in the Variable Expressions node there is the createCustomUrlPath function that you could try to build your input variable.

[Thiemo.Kellner]

I am switching to another node, however, I do not feel the vanilla file reader is appropriate as it requires column delimiter and such stuff. In fact, I wonder what the difference to the csv reader is. I go for the line reader and split the lines in the flow. I let you know what I achieve that way.

[Thiemo.Kellner]

My solution looks now as follows.