to read the parquet file from s3. every time i need to download it.
without download, it how to read it ?
i have many files , cant download all.
You could try and define an external table with the structure of your parquet files, I have built a workflow with Hive and a local big data environment
A short Google search seems to indicate there are external table options with S3 but I am not familiar with that. You will have to try a few things.
https://forums.aws.amazon.com/message.jspa?messageID=915638#915638
If you want to have the data on your system at a certain point you will have to load the data. Parquet would be beneficial because the files could be compressed.
Hi there @navinjadhav,
to my knowledge you can read parquet files without downloading them. Connect Amazon S3 Connection to Parquet Reader node directly.
Br,
Ivan
1 Like
This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.