Can't Get GPT Embeddings

KNIME Extensions
1

I was experimenting with the OpenAI extensions, and I wanted to perform the simple task of fetching some embeddings from GPT (as vectors), however I couldn’t figure out how to do this. The goal is to be able to use the vectors directly in clustering etc.

All the examples I found connect the “OpenAI Embeddings Connector” to a vector store. I instead attempted to connect this embeddings-connector to the “LLM Prompter” node (with the assumption that it might be able to return the vectors), but this didn’t work. So it seems that it can only be used for vector similarity search and not anything else.

Is there anyone that has had success doing this? It’s possible that I missed something obvious since this should be a trivial task. If not possible, I could just as easily use the REST API instead, but it would be nice to do it in a more ‘knimey’ way.

Thanks in advance to anyone who has any tips!

2

Hi there,

Here I let you one of my work flows to extract the embeddings to make your clustering. It works, I did something similar.

(Link to my Workflow)

Good look.

3

Hey @jacobcoles,
welcome to the KNIME forum!

I think that you are looking for the Text Embedder Node. You can connect your OpenAI Embeddings Model that you chose in the OpenAI Embeddings Connector as well as a table with texts that you want to embed. The Text Embedder will then embed each row with the specified model and return the vectors.

You can find a lot of example workflows following the KNIME for Generative AI Collection.

Best,
Alex

1 Like