The process of creating a Spark context (Livy) when executed on the server fails

[arcdevil]

Hello!

When executing a workflow locally, not a single error occurs.
But with remote start from the server, this process crashes with an error.
Error and node settings in the screenshots below.

Full process:

HDFS Connection:

Spart Context(Livy):

Error:

As I understood from the error, he tries to work with HDFS on behalf of the user starting the process, and not on behalf of the user specified in the node configuration.

[mareike.hoeger]

Hi @arcdevil
first of all: Welcome to the KNIME Community!
To your question:
Unless you have a secure cluster setup with impersonation, the Livy server will try to access HDFS with the user that runs the server, which seems to be admin in your case.
So you either have to let the server be run by the user livy or setup a secure cluster with impersonation. For the latter you find further information in the secure cluster connection guide: https://docs.knime.com/2019-12/bigdata_secured_cluster_connection_guide/index.html

best regards Mareike

[arcdevil]

Thanks for the welcome!

Please tell me, is this behavior only on the server?

It turns out that he ignores the authentication settings from the nodes and uses the username of the user who launched workflow? This is true?

[arcdevil]

Found understanding at the end of the document you indicated. Thanks!