@havarde maybe you can take a look at these examples using GPT4All and Ollama with KNIME nodes. For me the key to getting re-usable JSON results were the use of the instruct LLM model and experimenting with the prompts giving specific instructions.
There is an older post by @MartinDDDD about the use of structured output that might be worth looking at