Using Textfile on Knime-Server

KNIME Server
1

Hi @all,
how can I use textfiles (csv) on knime server. I’d like to put event-messages from my workflow in it and I won’t use a database-table (there are only few messages). I tried to use mountpoint and absolute path for the file, but I get the message: “File must be a local file but xxx seems to be a remote file”

I’m using Knime Server 4.5.2

thx in advance
Michael.

2

Hi Michael,

Are you using the CSV Writer node? What settings are you using? The CSV Writer doesn’t support appending to remote files (which also covers execution on KNIME Server), you can only overwrite. Maybe that is the issue?

Cheers,
Roland

3

Hi Roland,
exactly that’s the issue. The question is, what can I do, instead? Is it generally impossible to use textfiles for appending on knime server?

I even tried using sqlite - also without success…

bw,
Michael.

4

You should be able to use SQLite with a database connector using a knime:/ URL protocol. If you use the SQLite Connector node, you can use a url such as knime://knime.workflow/data/log.db

Steve

5

Hi Michael,

As Roland mentioned the ‘Append’ option isn’t supported when writing to files on the server with the KNIME protocol. I’ve opened a ticket to provide direct support for that.

In the meantime, you have a possible option as a workaround. One option is to use the absolute file path in the reader. That has a couple of potential issues:

  1. You might not know what the path is.
  2. If you move the server to use the new distributed executors functionality the workflow will no longer work. Probably not an issue yet, but will be in the future.
  3. Workflow won’t work without modifying the path when you move it to the local filesystem.

Unfortunately Steve’s suggestion also won’t work out of the box on the KNIME Server (KNIME protocol doesn’t support SQLite or H2 databases on the Server). Using a standalone DB like SQL Server or PostgreSQL will of course work, however I think that is not an option in your case.

Best,

Jon

6

Hi Michael,

I would suggest in this case to first read the file and than concatenate it with the new lines and finally overwrite the existing one.

This has the advantage that you can still use the workflow relative paths.

Best, Iris

2 Likes
7

Iris is exactly right here, her suggestion is definitely the best.

1 Like
8

OK, here is a workaround. Create your knime:// protocol url in a flow variable, e.g. in a Java Edit Variable (Simple) node:

return "knime://knime.workflow/data/log.db";

Then follow this with a URL to File Path (Variable) node. This will resolve the knime:// url to the absolute url. Then use that in the SQLite Connector node:

image

Steve

9

@Iris: thx, this ist currently good for me, because there will be just a few hundreds of lines
@jonfuller: thx for your support. using standalone DB is in fact no option.
@s.roughley: thx, I tried that, but I failed…

bw,
Michael.

1 Like
10

Hi Steve,
thanks für your hint - I tried that several times, but it does not work. I suppose, the reason is in our data center. They’re using a large Storage Area Network for Knime Server…
So, I have to use a separate table on our database server…

Michael.