EMR Spark Connection Error (Livy)

#1

Hello,

I am trying to connect Knime Desktop to my EMR Spark cluster, but I am constantly getting the following error when I am trying to create the spark context.

Execute failed: Failed to create Livy session. Please consult the Livy server log. (Exception)

Here is my Spark cluster info,
Release label: emr-5.9.0
Applications: Hue 4.0.1, Spark 2.2.0, Zeppelin 0.7.2, Livy 0.4.0

The livy server logs are also attached to this topic.
livy_logs.log (7.8 KB)

Please advice.

0 Likes

#2

Hi @sguleria,
welcome to the community.
Could you also take a look into the log files of the failing application?
In this case: application_1557152545038_0039
Do you use the S3 Connector for the Remote File System in the Livy node?

best regards Mareike

0 Likes

#3

Yes, I am using S3 Connector for the FileSystem. I was able to connect to the EMR cluster yesterday upon starting a new EMR cluster, but once the connection terminated I was not able to connect again.

I will try to get the logs of failing application.

0 Likes