Hi, seeking an advise from expert on how to do similarity search. I want a function where user can search anything, if it has the same meaning it will show same answer.
Example.
Set 1: What are some popular tourist attractions in Paris, France?
Set 2: Which famous landmarks should I visit when traveling to Paris, France?
to be honest I don’t think that is what the Table View is capable of by default.
When you say you want to search for “meaning” it sounds somewhat like a potential use case for an App / Component that leverages KNIME’s Generative AI integration:
From a high-level point of view this would require:
Using embedding models to vectorize your data and storing the vector / data pairs in a vectorstore
creating a user interface using widgets / component composite views to obtain user input. User input is then also vectorized and a vector search is performed on the data in the vectorstore
the “most similar” search results (to the user input) are obtained and displayed to the user (that could be what is displayed in a Table View
As you can probably guess this is not a “light-weight, fast-to-realise” topic…
To get you started I recommend to install that AI extension and to check out the LLM-related examples on the KNIME hub:
Also @mlauber71 wrote an article on embeddings etc. including KNIME:
@WanRashid you might want to check some examples of how to have a dialogue in a knime component with you own data using local LLMs in this collection. I assume that what you mean by similarity search could be what a vector store does.
And the accompanying articles:
If you are fine with also using OpenAI you can check the KNIME space:
