How to give a large amount of data to the request body in REST put and post nodes

KNIME Extensions
1

We have a need to use REST nodes for populating into a database (triple store).

We want to put a large block of data (several million triples) as a block through the “body” of a PUT or POST request.

We have tried to do this by using a group by node to concatenate the rows of a single column table (the entry in each cell is a triple), and to put either by specifying the request body column or by converting to a flow variable and assign to a constant body.

In both cases it fails when we join a large number of rows together.

Is there a way to have the REST node take a table into the body as one call, without making separate calls for each row?

Best

2

Hi,

depending on the REST endpoint you’re using and what kind of format it expects you should be able to format your data in such a way that you can use a single request. If the endpoint for example supports JSON you could use the Table to JSON node to create a single JSON cell containing the whole content of the table.

Out of curiosity: is there any reason why you don’t use the new database integration (Node Repository -> DB)?

Cheers,
Moritz

2 Likes