How to specify filename in Spark to csv node

Hi all,

In the settings, it seems that you can only set up to the folder,
but is there a way I don’t know about?

Result I expected :

Result :

Hi @jjlee,

Spark writes every partition in parallel, and creates an output directory with one file per partition.

If you have only a small amount of data:

  • You can limit the partition count on the Partitions tab in the dialog, this creates only one output file.
  • You can use the Spark to Table and CSV Writer node.

Cheers,
Sascha

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.