I am trying to create a workflow that would use local LLMs like Llama32 with the help of Ollama and the KNIME AI nodes as described in the article by @roberto_cadili How to leverage open source LLMs locally via Ollama . [image] I tested several local URLs and models but I always receive the same error message ‘invalid input’. I wonder if there is a problem with the current implementation - Ollama version 0.3.14. Maybe someone can also try and give feedback. [image] Ollama KNIME - Creating Vector Store – mlauber71 Create a Vector Store from a PDF and query it with local Ollama instance you need Ollama installed and ready to run on your machine tested it with Llama 3.2 Em… What does work is creating such Vector stores with GPT4All and the KNIME nodes ( Chroma and FAISS and then using it with Ollama).

Hi @mlauber71 , thanks for checking out the blog post :slight_smile:. What you’re trying to do is unfortunately not possible using the OpenAI Authenticator node. In Step 2 of the blog post, I also explained why: … Next, we drag and drop the OpenAI Chat Model Connector node, which we can use to connect to Ollama’s chat, instruct and code models. We need this specific connector node (and not the OpenAI LLM Connector) because Ollama has built-in compatibility only with the OpenAI Chat Completions API , which is what the OpenAI Chat Model Connector node uses behind the hood. If we wish to leverage Ollama’s vision and embeddings models, we can do so using the nodes of the KNIME REST Client Extension . To connect to embedding models available via Ollama, you can use the POST Request node (Mind: make sure that the model you want to use is compatible with the task of creating embeddings). To show how this would work, a couple of months ago I built a workflow to connect to embedding models and vision models from Ollama. Find it below: [image] Leverage open-source, local LLMs for vision and embeddings via Ollama –... Leverage open-source, local LLMs for vision and embeddings via Ollama This workflow shows how to leverage an LLM for vision and embeddings available via Ollama… Hope this helps :slight_smile:. Happy KNIMEing, Roberto

Ollama / OpenAI - local embeddings not working

KNIME Analytics Platform

ScottF October 29, 2024, 2:38pm 11

Hi @tescnovonesis -

From a different thread: