Google Big Query: "Exceed rate limits" - How to upload large tables

Hi aherzberg,
you don’t need to tick the Append JDBC parameter to URL option since the Simba driver also accepts parameters via the standard property object and then seems to have problems with the authentication parameters. Also as far as I understand the description of the parameter EnableHighTroughputAPI and HighThroughputActivationRatio they only improves query performance since it uses the storage API for Result Sets.

In general Google Big Query has a lot of restrictions for DML (insert, update, delete) statements. For example standard tables only allow 1000 DML operations per day.
The only advice I can give you right now is to use the DB Loader node in KNIME Analytics Platform to upload mass data which uses another API. However this only supports loading of data but not updating. So please contact the Google BigQuery support maybe there is a way to increase the standard limitations.
Bye
Tobias

4 Likes