The new reader nodes (CSV & Excel) all default to only scan 50 rows to determine the data type. I find this fails nearly every time and have to manually turn this off each time I use one of these nodes. Is there anyway to change the default to either not limit the rows or at least use a larger number (>1000 rows), so I don’t have to manually make the change every single time?
3 Likes
We already increased the default and it will be shipped with the bug fix release. Is there a magic number that applies to your files?
Best
Mark
~1000 rows would work 90% of the time for me, 10,000 would work 99% of the time. Either case takes less than a second to scan with ~20 columns of data
2 Likes
I have the exact same issue. Feels clunky having to disable every time. Is it possible to either default to not limit the data rows scanned or add a preference to auto-enable or not?
Thank you for fixing this! Things run much smoother after the update.
2 Likes
This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.