Response error as "!!!!!!!!!!!!!" for the gpt4all model

KNIME Analytics Platform
1

@mlauber71 please do help

Screenshot (1)
Screenshot (1)1280×649 96.7 KB

I am getting the response as “!!!” in Create your own LLM Vector Store Workflow
i dont know where is the problem
I also have an other problem
in GPT4all- chat DataApp workflow, in GPT4all chatmodel connector node i am facing an issue. It says no file found at path and i am not able to find falcon-q4_0.gguf in gpt4all

2

Have you downloaded the model gguf file? GPT4all can only use the models you download
br

1 Like
3

@intern_ajay welcome to the KNIME forum. I think I have seen this problem somewhere. You might want to download the example again and make sure you have the latest version of GPT4All installed and you might have to download (and select) a .gguf model again.

Also make sure you download the entire workflow group. The folder for the .gguf should be present. The selection should look something like this (depending on which models you try to use):

image
image2086×1014 218 KB

What operating system are you using?

1 Like
4

Indeed has been mentioned before :wink:

5

I am using windows server
I downloaded the whole workflow group again and gpt4all is latest version and updated
I imported the workflow group to my knime workspace
I downloaded the all-MiniLM-L6-v2-f16.gguf model to the gpt4all_models folder
still I am getting the same error in the response as “!!!”

6

@intern_ajay I have no immediate idea what is going on. You could maybe check all the settings if you see something Also a fresh logfile in Debug mode might give some insights. Also you might want to check the amount of memory allocated to KNIME.

Would GPT4All as a standalone tool work? Can you check the settings there if it tries to use a GPU or maybe try to use one.

it might be possible that there are changes with GPT4All (again) that make problems with the nodes. We will have to see.

7

Screenshot (2)
Screenshot (2)1280×649 50.2 KB

Hi,
the response error is still the same " !!!" but when i run the chatapp i am getting output from the existing knowledge of the LLM and not from the documents in the document folder
but I need the ChatApp to do RAG
Pls help

8

If you query only the vectorstore do you get an answer?
br

9

I am getting answer but its a generic answer. Its not using the retrieved list from the documents

10

It seems there is a new version of gguf that is not currently supported by the KNIME nodes - there is a ticket to fix that: AP-22323.

Also it will make sense to check the correct prompts templates of a model in order for it to work. They might differ between different LLMs.

image

11

I would appreciate if KNIME would “wrap” the template for the specific model automatically like some other tools
br

1 Like