Spark join behavior

[Fabien_Couprie]

Databricks 3.2 :
the joiner configuration :

return the error : ERROR Spark Joiner 3:655 Execute failed: No columns selected

But Pyspark 2->1 with resultDataFrame1 = dataFrame1.join(dataFrame2,dataFrame1[‘email’]==dataFrame2[‘Emailkey’],“inner”)

return the result as expected.

Best regards

[sascha.wolke]

Hi @Fabien_Couprie,

the second screenshot has only one column in the right table. This is the join column, and that’s why the node complains about an empty columns list. Maybe the table contain unsupported columns?

Cheers
Sascha