Announcing KNIME Analytics Platform 5.2

[mlauber71]

@TosinLitics I do not have extensive experience with Document metadata but you might try to add them using the Document Data Assigner and provide them to the FAISS Vector Store Creator. Maybe it is worth experimenting with that.

(from this example)

Creating a Local LLM Vector Store from PDFs with KNIME and GPT4All

https://hub.knime.com/-/spaces/-/~WNe6bb2w2bemYBWE/current-state/

Also you could experiment with the ‘window’ size of texts and documents you are providing:

correct relative path for GPT4All workflow (GPT4All - Create your own LLM Vector Store)

@mgirdwood I have this example where I load several PDFs from the author Stefan Zweig into a Vector store and make them accessible. You might have to experiment with the meta data of the documents and also with the ‘window’ you give the model. There is a component that tries to do a more complex preparation of the data with giving blocks of text (since a novel might benefit from a few sentences of context). You might have to toy around with such concepts for your own data. A coffee machine ma…