Remove rows that are contained (as sub-strings) in other rows

[saqib]

Hello all,

I have a table like as follows:
This is a cat
This is a pet
This is a dog
is
a
is a
is
is a dog
That is a dog

I would like to end up with a table as follows:
This is a cat
This is a pet
This is a dog
That is a dog

Essentially remove the rows that are already contained (as sub-strings) in other rows.

Thanks

[armingrudd]

Hi,

Considering the examples, you can use a Rule-based Row Filter node and this expression:
$column$ LIKE “* is a *” => TRUE

Best,
Armin

[ipazin]

Hi there!

I think @saqib is looking for general approach to his problem. Maybe a bit clearer requirements and possible input could help others suggest a solution Anyways I saw you got some answers here which seem pretty good so I will make assumption you got what you were looking for

Br,
Ivan

[saqib]

@ipazin,

Yup I did get some solution on Stackoverflow, but they don’t seem to scale for large datasets. I am working with a large dataset, so I am looking for something that more optimized.

Any ideas?

Thanks.

[izaychik63]

If you have frequency information on the expected correct phrases then this method can be used

Renaming a string in a row based on another column

Hello all, i am working on a small project at school. Actually i am at one point where i have no clue how to go further. I have a column with company names where due to spelling mistakes the company name can be spelled wrong. For example the company name has 2 instead of 1 space, a letter is missing or the dot notation is wrong. In this case i have done a count on the company names and the one with the most hits is the right name. I would like to change all the other columns that are similiar …

[saqib]

@izaychik63,

I don’t have the frequency information.

Thanks,
Saqib

[ipazin]

Hi there,

I would first try with workflow optimization rather than a new idea/approach. This is a great blog post about it: https://www.knime.com/blog/optimizing-knime-workflows-for-performance

Additionally KNIME summer release (3.8) should bring faster workflow execution so count on that as well

Br,
Ivan

[acommons]

I think this might get you started. Not a complete solution but I think the start of one, in fact very close to one…Hint: Just add more Regex the filter.

Testing1.knwf (17.7 KB)