I’d like to read in seven days of CSVs from an S3 bucket and append the raw data read-in to prior weeks. I’ve seen 3-4 prior conversations on this topic but none of them have a good example workflow to learn from and apply to this problem.
Additional Issue…what do you when a variable Loop End requires the ‘allow variable column types’ due to a column being a string in one file and double in another? This variable column type is preventing the option for using a CSV Writer to save/append the new data to an exisiting CSV.
Hoping a screenshot will further explain the context of this problem:
@izaychik63 I like you suggestion to use SqLite DB but can’t get into something unfamiliar right now. Looks like it’s worth exploring though, do you have a good suggested starting link resource you can share?
@mlauber71 to the rescue again…I followed your prior post on the readr suggestion you made to someone else and utilized their recommendation to ignore column headers so all is imported as string to start with.