Azure AI connectors

KNIME Analytics Platform
1

Hi!

Im trying to build a duplicate detection model with Knime And azure AI, but cant figure out the correct nodes.

I have a list of 6000 company names, that have been written slightly in different format or in different language, but in reality is the same company. If I try to use the normal AI authenticator-Open ai chat model connector-chat model prompt, and feed the data as json to the model, the response is cut off, perhaps due to the request response (using GPT4).

Somewhere I read that using the enbeddings connector with text enbedding ada might be the correct approach, but I cant figure out how to pass the data and the question to the model as no other node will connect to this one.

Do you have any suggestions on how to get this done with any of the Open ai nodes :)?

2

Hello @Data_processor,

Welcome to the community.

Did you try increasing the Maximum Response Length (token) in your Connector node?

Here is a simple example: I use a local embedding model to create embeddings for unique company names and store them in a vector store. Then, I create a prompt to retrieve the most similar company name from the vector store using the retrieval node.

Is this the kind of approach you’re looking for? If do not have the unique list of companies this might not be possible.

Best,
Keerthan