I have a JSON API that requires “GET Request” for the authentication. Is there a limit on the amount of data returned? I potentially will have millions of rows of data returned (yes, millions…). I would “assume” that being simply text this shouldn’t be a problem, but I fear life may not be so easy.
I saw a post talking about 4096 being some sort of “chunk size”, but I’m not sure I “get it”, regarding limits.
More specifically, my node-path is (also see uploaded screen-shot):
- “Get Request” to
- “JSON Path” to
- “Ungroup” to
- “JSON to Table” to
- {some TBD approach of saving output to Excel or SQL Server}
I’m hoping there are no limits/maximums through the “JSON to Table” node above. Naturally, Excel would have notable limits (though I’ll generally use SQL Server).
If there is no general restriction in the API, this should be possible. There is no limitation in the KNIME nodes (GET Request, JSON to Table) except your memory.
