JSON Writer Single File

Feedback & Ideas
1

Writing JSON files feels rather cumbersome if you load one JSON file, want to make some changes and save it back (locally).

JSON writer requires a flow variable that is “one level above” the path flow variable created from JSON reader, and the file name in an additional column.

JSON Row Combiner and Writer, similar to some of the other nodes, force the creation of an array per row. a merge option is not offered (which obviously would need to fail if a key is duplicated).

having a patch and applying it directly on a local JSON file is not available either.

if you convert a JSON to a table first and add additional columns, you need to convert it back and then have the same issue with the Path requiring a folder.

to bring an additional (arbitrary) example:

you have a local JSON config file like

{
    "created_at" : "2011-11-11T11:11:11",
    "SQL" : {
        "Server" : "serveraddress",
        "Port" : 1234,
        "Database" : "dbname"
    },
    "Name" : "Project Name"
}

and you want to add or modify some information and save it back

{
    "created_at" : "2011-11-11T11:11:11",
    "SQL" : {
        "Server" : "serveraddress",
        "Port" : 1234,
        "Database" : "dbname"
    },
    "Name" : "Project Name",
    "Retention" : {
        "archive_on" : "2012-11-11T11:11:11",
        "delete_after" : "2016-11-11T11:11:11"
    }
}

(yes, I am aware that JSON spec isnt trivial and allows for many different ways of handling, but updating a single file should be made easier)

2

Hey @fe145f9fb2a1f6b,

thanks for bringing this up. Wouldn’t the existing “JSON Transformer” node solve this use case?

Update:
Just tried it and indeed the “JSON Transformer” could be used. I still agree that a merge operation or dedicated merge node would be preferrable as understanding the json transfomer is surely not an easy task

JSONTransformer.knwf (49.8 KB)

Greetings,

Daniel

3

@DanielBog please read again.

the post is about writing back to a single local file.
the transformation part is secondary.

4

Got it and makes a lot of sense.

I guess the JSON writer was never designed to be used in such a way.

How would you expect it to work? JSON writer has a mode selection between single and multiple files :thinking:

Greetings,

Daniel

1 Like
5

the way its designed right now, it will always produce 1 file per row.

it would be a lot better to have something like a

JSON (Column) Writer which does what the current version does
JSON Table Writer which takes a whole table, converts and writes it into one file

the first could contain a toggle to swap between current behavior and Write first row only which then takes a PATH (flow var) instead.

other options possible too but the current one just doesnt fit.

1 Like