# Job Finder Chatbot : Replacing OpenAI with GPT4All

**URL:** <https://forum.knime.com/t/job-finder-chatbot-replacing-openai-with-gpt4all/89058>\
**Category:** KNIME Analytics Platform\
**Created:** [July 10, 2025, 11:17pm UTC](https://forum.knime.com/t/job-finder-chatbot-replacing-openai-with-gpt4all/89058 "2025-07-10T23:17:44Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Foxyellow](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/f/9f8e36/32.png) [@Foxyellow](https://forum.knime.com/u/Foxyellow)\
**Post date:** [July 10, 2025, 11:17pm UTC](https://forum.knime.com/t/job-finder-chatbot-replacing-openai-with-gpt4all/89058/1 "2025-07-10T23:17:44Z")

</div>

Hi everyone,

I’m currently working on reproducing the KNIME workflow from the blog post, [how-to-build-custom-ai-powered-chatbot](https://www.knime.com/blog/how-to-build-custom-ai-powered-chatbot)  
but with a key constraint: I want to **use only open-source models** , without relying on OpenAI services.

Let me summarize my progress and the problem I’m facing:

### Step 1 – Vector Store Creation with Open-Source Embeddings

I was able to **import a dataset of job offers** , and extract **embeddings** of the job descriptions using the **GPT4All Embeddings Connector** node with a local embedding model. I tested several models, and some didn’t work for embedding generation, but eventually I managed to generate valid embeddings and create a **FAISS vector store** using the **FAISS Vector Store Creator** node.

 ![image](https://forum-cdn.knime.com/uploads/default/original/3X/4/9/499c675a4d8361e09f63cde91806e54544141e8e.png)

### Step 2 – Creating the Agent with Open-Source Chat Model (Problem)

This is where I’m stuck.

According to the workflow, the **Agent Prompter** node must be connected to an agent created by the **OpenAI Functions Agent Creator** node. However, I can’t use this node with a GPT4All chat model — it only supports OpenAI nodes.

I tried using the **GPT4All Chat Model Connector** with the **Agent Prompter** node, but the nodes are not compatible.

```auto
The provided GPT4All Chat Model is incompatible with the expected type Agent. Connect one of the following nodes: OpenAI Functions Agent Creator.

```

What I’m trying to achieve:

- A local conversational agent (e.g., with GPT4All) that queries the FAISS vector store and returns relevant job suggestions to the user.
- A user-friendly UI (eventually) to interact with the chatbot.

 ![image](https://forum-cdn.knime.com/uploads/default/original/3X/5/d/5da89296f04aaf5cd49bbc130480677a13bac2df.png)

My questions:

- **Is there any way to build a working conversational retrieval agent in KNIME using GPT4All instead of OpenAI models?**
- **Are there alternative nodes or techniques to simulate agent behavior (like query + generation with context) with open-source chat models in KNIME?**  
Thanks a lot for your help!  
I’d love to complete this without OpenAI and showcase what can be done with entirely local open-source models in KNIME.

Let me know if you’d like me to share the workflow file.

Best regards,

---

<div class="post-metadata">

**Author:** ![MartinDDDD](https://forum-cdn.knime.com/user_avatar/forum.knime.com/martindddd/32/69988_2.png) [@MartinDDDD](https://forum.knime.com/u/MartinDDDD)\
**Post date:** [July 11, 2025, 4:31am UTC](https://forum.knime.com/t/job-finder-chatbot-replacing-openai-with-gpt4all/89058/2 "2025-07-11T04:31:20Z")

</div>

Hey there,

Amazing to see you taking on such a bigger project!

I did some research as to why you may be struggling with using Function Agents in GPT4all. I think by now there are a lot of models - also open source - that support function calling.

The issue seems to be that GPT4all API is in general OpenAI compatible, but does not support function calling.

If you want to stay local, maybe use Ollama? Although I think Ollama does not support embeddings API of OpenAI so you’d have to embed via GPT4All and function call via Ollama, which could mean loading two models in parallel into GPU memory…

---

<div class="post-metadata">

**Author:** ![mlauber71](https://forum-cdn.knime.com/user_avatar/forum.knime.com/mlauber71/32/73590_2.png) [@mlauber71](https://forum.knime.com/u/mlauber71)\
**Post date:** [July 11, 2025, 7:13am UTC](https://forum.knime.com/t/job-finder-chatbot-replacing-openai-with-gpt4all/89058/3 "2025-07-11T07:13:37Z")

</div>

@Foxyellow I am working on [modifying a workflow](https://hub.knime.com/s/UQlKPMIPSy5IKWjm) by @MartinDDDD to use Ollama and agents but at certain points the transfer of KNIME workflows into tools fails. I will have to try.

Meanwhile you can explore my collection on the KNIME hub and see if it does help you. There are also some efforts to use [CrewAI](https://hub.knime.com/s/nx2kD4C8Jwu0ZasW) - (with KNIME 4 for that matter). Also borrowing from one of Martin’s articles …

> **[mlauber71/LLM\_Space](https://hub.knime.com/mlauber71/spaces/LLM_Space/~17k4zAECNryrZw1X/)**
>
> Some KNIME workflows to test the use of LLMs (large language models). Also using local GPT4All models.
> Download the whole workflow group..
> 
> The GPt4All are optimized for KNIME 5.3+. The Ollama and CrewAI examples will also work with KNIME 4.7 –...

This one has local Ollama usage. Vector stores will have to be[created with GPT4All](https://hub.knime.com/s/8sXYiWAYINWF2Alz) since the KNIME Ollama nodes do not support this (yet?).

> **[Ollama KNIME - Creating Vector Store – mlauber71](https://hub.knime.com/mlauber71/spaces/LLM_Space/Ollama%20KNIME%20-%20Creating%20Vector%20Store~8sXYiWAYINWF2Alz/current-state)**
>
> Create a Vector Store with GPT4All from a PDF and query it with local Ollama instance you need Ollama installed and ready to run on your machine, also you shou…

---

<div class="post-metadata">

**Author:** ![mlauber71](https://forum-cdn.knime.com/user_avatar/forum.knime.com/mlauber71/32/73590_2.png) [@mlauber71](https://forum.knime.com/u/mlauber71)\
**Post date:** [July 11, 2025, 11:13am UTC](https://forum.knime.com/t/job-finder-chatbot-replacing-openai-with-gpt4all/89058/4 "2025-07-11T11:13:56Z")

</div>

@Foxyellow I adapted @MartinDDDD workflow using [Ollama](https://medium.com/low-code-for-advanced-data-science/llama3-and-knime-build-your-local-vector-store-from-pdfs-and-other-documents-237eda761c1c) and local LLM [Qwen](https://ollama.com/library/qwen2.5:32b) supporting tools.

 ![image](https://forum-cdn.knime.com/uploads/default/original/3X/c/8/c8aefda4d57bc6f37aacea47798ce750b9f3b4c0.png)

the question is answered correctly. It would be interesting to put that in a chat context which I think could be done and also to provide the result in a structured way ([JSON](https://medium.com/low-code-for-advanced-data-science/chat-with-local-llama3-model-via-ollama-in-knime-analytics-platform-also-extract-logs-into-aca61e4a690a)) so as to continue with a value.

 ![image](https://forum-cdn.knime.com/uploads/default/original/3X/d/1/d18546682372789a6edfc7570a0d73f3b8a4aa84.png)

To be completely honest I do not fully understand how these tools are handled and how to turn KNIME workflows into tools in general. This really sounds like magic 🙂 I think I will have to investigate further (or [maybe read the article](https://medium.com/low-code-for-advanced-data-science/ai-agents-in-knime-5-5-933ac54dca84) …).

> **[AgenticFeatures – mlauber71](https://hub.knime.com/mlauber71/spaces/LLM_Space/Basic%20Agents%20-%20by%20martindddd/AgenticFeatures~z6LpIGOsgew0n8nm/current-state)**
>
> Solutions for data science: find workflows, nodes and components, and collaborate in spaces.

---

<div class="post-metadata">

**Author:** ![MartinDDDD](https://forum-cdn.knime.com/user_avatar/forum.knime.com/martindddd/32/69988_2.png) [@MartinDDDD](https://forum.knime.com/u/MartinDDDD)\
**Post date:** [July 11, 2025, 11:43am UTC](https://forum.knime.com/t/job-finder-chatbot-replacing-openai-with-gpt4all/89058/5 "2025-07-11T11:43:24Z")

</div>

> [@mlauber71](#):
>
> To be completely honest I do not fully understand how these tools are handled and how to turn KNIME workflows into tools in general

I think once you try it you will be surprised how easy it is with 5.5…

Take any workflow, parameterise it using configuration nodes, give the configuration nodes meaningful descriptions, create a meaningful workflow description and save to the tools folder… that’s it…

If it didn’t “click” after the first article, then read the second one as well 🙂

> **[AI Agents in KNIME 5.5 Part 2](https://medium.com/@martindieste/ai-agents-in-knime-5-5-part-2-eded98a9f57b)**
>
> Expanding our Agents capabilities with more tools! — Check Part 1 for an intro

---

<div class="post-metadata">

**Author:** ![mgirdwood](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/m/a5b964/32.png) [@mgirdwood](https://forum.knime.com/u/mgirdwood)\
**Post date:** [July 11, 2025, 12:40pm UTC](https://forum.knime.com/t/job-finder-chatbot-replacing-openai-with-gpt4all/89058/6 "2025-07-11T12:40:15Z")

</div>

Hi @Foxyellow  
MartinDD and MLauber have been my “Go To” on these topics for a while now.

Just to add to the options on local LLMs. A short while ago I switched from using the Ollama and GPT4all option and started using the LM Studio App. As its a GUI i have found it more friendly than Ollama and there seem to be more models available than for GPT4All.  
I use the standard OpenAI nodes and its quick and easy to use I have found.  
Just tried the Agentic features (tools) workflow from Martin that MLauber mentioned above and it ran no issues

---

<div class="post-metadata">

**Author:** ![mlauber71](https://forum-cdn.knime.com/user_avatar/forum.knime.com/mlauber71/32/73590_2.png) [@mlauber71](https://forum.knime.com/u/mlauber71)\
**Post date:** [July 11, 2025, 12:55pm UTC](https://forum.knime.com/t/job-finder-chatbot-replacing-openai-with-gpt4all/89058/7 "2025-07-11T12:55:40Z")

</div>

@MartinDDDD when adapting to the LLM Qwen I constantly get configuration issues with variables. Question is if there is a logic or syntax that can be adapted to make the use run smoothly.

---

<div class="post-metadata">

**Author:** ![MartinDDDD](https://forum-cdn.knime.com/user_avatar/forum.knime.com/martindddd/32/69988_2.png) [@MartinDDDD](https://forum.knime.com/u/MartinDDDD)\
**Post date:** [July 11, 2025, 1:08pm UTC](https://forum.knime.com/t/job-finder-chatbot-replacing-openai-with-gpt4all/89058/8 "2025-07-11T13:08:59Z")

</div>

So the agent tries to use a tool, but screws up the inputs?

I think that if you run a 7bn parameter model locally it may not be performing that well with function calling… can you add Message Part Extractor? That should show you the JSON that the agent is trying to use on each tool…

---

<div class="post-metadata">

**Author:** ![mgirdwood](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/m/a5b964/32.png) [@mgirdwood](https://forum.knime.com/u/mgirdwood)\
**Post date:** [July 11, 2025, 4:21pm UTC](https://forum.knime.com/t/job-finder-chatbot-replacing-openai-with-gpt4all/89058/9 "2025-07-11T16:21:05Z")

</div>

@Foxyellow Just adding here that as I have just discovered, LM Studio and OpenAI nodes work fine with the normal nodes However the combination doesnt work for Embeddings and you still need to use the GPT4All node - **But** you can point the GPT4All node to the LM embedding model meaning i can still use LM Studio for models and dont need to install GPT4All

---

<div class="post-metadata">

**Author:** ![system](https://forum-cdn.knime.com/uploads/default/original/1X/ab3ccf34482a0329361734a18199390177204f15.png) [@system](https://forum.knime.com/u/system)\
**Post date:** [October 9, 2025, 4:21pm UTC](https://forum.knime.com/t/job-finder-chatbot-replacing-openai-with-gpt4all/89058/10 "2025-10-09T16:21:32Z")

</div>

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.
