[Livy] Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources

Hi, I’m trying to resolve this error.

I’m using a configuration of one spark master server and three worker servers connected to a knime ap.

The workflow I am running is as follows.

When tested, the HDFS connection and the LIVY connection work fine.
However, the problem is when I have multiple people connecting to the same server at the same time to run the workflow.

Taking the above workflow as an example, it works fine up to the previous node, but then parquet to spark gets infinite loading and doesn’t run properly.
The above error occurs when someone else is connecting to livy first, otherwise it connects without infinite loading.
The result is, only one person can connect to livy

And here are the logs from livy when the above error occurs.

Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources

How can I solve this error?
Please guide me.

Hi @yun08,

That usual means, there are no more resources available to run another job on your cluster. Maybe double-check your workers are connected, the spark context use them, and the initial job does not consume/block all resources. You can find more details in the YARN resource manager and Spark WebUI via the Livy frontend.

Cheers,
Sascha