NoClassDefFoundError: BouncyCastleProvider when creating Spark context with KNIME 5.4.2 (but not in KNIME 4.6)

KNIME Extensions
1

Hi everyone,

I’m currently encountering an issue when trying to create a Spark context using the Create Spark Context (Livy) node in KNIME 5.4.2. The same workflow works perfectly in KNIME 4.6, but in the newer version I receive the following error:

Execute failed: org/bouncycastle/jce/provider/BouncyCastleProvider
java.lang.NoClassDefFoundError: org/bouncycastle/jce/provider/BouncyCastleProvider
…
Caused by: java.lang.ClassNotFoundException: org.bouncycastle.jce.provider.BouncyCastleProvider cannot be found by org.apache.hadoop.client_3.4.0.knimebd-20241108

This happens when connecting to WebHDFS with SSL, and from what I understand, it’s related to a missing dependency: the Bouncy Castle security provider. It seems this is required by the Hadoop libraries for encryption and secure communication when accessing HDFS.

My guess is that KNIME 5.4.2 no longer includes this library by default, while KNIME 4.6 did.

Questions:

  1. Has anyone else experienced this issue when migrating workflows from KNIME 4.x to 5.x?

  2. Is there a recommended way to manually include the Bouncy Castle JAR (e.g., via the dropins/ folder or through a specific plugin)?

  3. Could this be related to changes in how KNIME packages the Hadoop/Spark integrations in version 5.x?

Any help or insights would be greatly appreciated!

Thanks in advance,
And

2

HI all,
maybe a compatibility problem on hadoop we have:
/opt/cloudera/parcels/CDH-7.1.9-1.cdh7.1.9.p6.51045883/lib/hadoop-yarn/lib/bcprov-jdk15on-1.68.jar
on Knime bcprov_1.77.0.jar

Can I change this Jar in Knime?
thank a lot fot attention
Have a nice time