Validating Values using AI

[mmrostami]

Is there a workflow to use AI to check and validate column values of a dataset? For example, let’s say a dataset with a “Country” column, but some country names misspelled, or they appear in different or the entries are not country names but rather names of territories or states. I want to use AI to check this for me. HOW?

[CarlosEnrique84]

Hi mmrostami,

The following is not exactly what you are looking for but maybe it could help you…

Cheers,

Carlos

Fuzzy Matching of Strings – knime

[mlauber71]

@mmrostami you could indeed try a fuzzy match if you have a list of all correct spellings.

AI for discovery similar terms

@denisfi maybe you do not need fancy AI but just some sort of similarity match in a loop. [image] You can adapt the number that you will accept as ‘similar’ [image]

You can also try and use a (local) LLM to let it clean and detect the correct names. You might have to see how to instruct the model to just give back the results and no big speech.

Splitting fields with Multiple Values

@TSL4000 welcome to the KNIME Forum. I thought this would be a fun task for KNIME’s LLM nodes and Ollama/Llama3.2. The result after some tweaking of the input and the prompt can look like this: [image] Some lines have not been fully covered … mainly where there are several names but no and or ampersand. One might have to amend the prompt to cover such cases but first: What is happening is that the original content gets converted in a JSOn structure in chunks of 10 and this JSON then gets fed…

[rfeigel]

Here’s a variety of approaches you can try.

Join w Partial Information – rfeigel