How to configure the "CSV to Spark" node?

KNIME Extensions
1

Hi all,

I’ve got a few questions regarding the “CSV to Spark” node.

I need to load a normal CSV file, but there’s no chance to see the list of files in the S3 directory I use (the S3 connector has been correctly activated).

I read in the description of the node that I can choose between file and directory, through the mode option. But I can’t see such an option in the configuration page (see picture below).

Fig. 5
Fig. 52238×860 162 KB

The only working option is to call a directory, but when I run the node I get two types of error.

  1. If I simply insert the directory in which I have the CSV file on S3, this is the message I receive:

Fig. 2
Fig. 21630×1396 79 KB

  1. If I enter the spark directory created in S3, this is the other error message:

Fig. 3
Fig. 31640×1414 87.4 KB

Using a common “CSV reader” node, the file is correctly loaded from S3, as you can see in the picture below:

Fig. 4
Fig. 41124×978 53.7 KB

In the following (last) picture you can see the parameters of the “CSV to Spark” node:

Fig. 1
Fig. 11662×1416 165 KB

So where’s the problem? Why I can’t read a CSV file? Any clue?

Thanks for your patience,
Alberto

2

Hi @alberto_trobia,

As far as I remember, you have to specify the bucket name in the path, something like /my-bucket/perspark/ in the CSV to Spark node. Does this work? If not, the console might contains more details about the Problem.

Cheers,
Sascha

3

Hi Sascha, and thanks again for your help!

“perspark” is the name of the bucket, the one that contains the csv file. The bucket has been selected through the “browse” button.

What do you mean by “console”? I’m afraid I can’t read a console. What am I supposed to search for?

Thanks,
Alberto

4

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.