The EMR cluster has the permission to read/write data. I re-checked it using Spark to Parquet node. The output path for saving the model is the same as the one used to save the test data file.
It is working fine on the test data but not on the models. I am not sure if the output path is the issue.
This is the output path which I am using
s3://‘my-bucket-name’/s3-insightsmax-orch/NBA_Output_folder/Production/ModellingScenariosResults/ModelTrainStructure/
The code which I am using to save the model:
cvModel.write().save(output_path)
I even run the code without write function but it showed the same error