# Duplicate Row filter

**URL:** <https://forum.knime.com/t/duplicate-row-filter/21645>\
**Category:** KNIME Analytics Platform\
**Created:** [March 10, 2020, 2:18pm UTC](https://forum.knime.com/t/duplicate-row-filter/21645 "2020-03-10T14:18:03Z")\
**Posts on this page:** 1\
**Showing post:** 4

<div class="post-metadata">

**Author:** ![mlauber71](https://forum-cdn.knime.com/user_avatar/forum.knime.com/mlauber71/32/73590_2.png) [@mlauber71](https://forum.knime.com/u/mlauber71)\
**Post date:** [March 10, 2020, 10:45pm UTC](https://forum.knime.com/t/duplicate-row-filter/21645/4 "2020-03-10T22:45:45Z")

</div>

I can add one more idea to use SQL and row\_number if you have an ID that would indicate a duplicate entry.

```auto
SELECT * FROM 

( SELECT * 
, row_number() over (partition BY `ID` ORDER BY `date` DESC) AS `rank_id`
FROM `default`.`data1`  
) t1
-- only keep the entry that will be 
-- left on top of a group of identical v_name
WHERE `t1`.`rank_id` = 1
;

```

> [@Remove rows with duplicate values](https://forum.knime.com/t/remove-rows-with-duplicate-values/11105/15):
>
> @serendipitytech I can add one more idea to remove duplicates and ensure unique IDs using SQL (Hive in this case) and row\_number() if you have to deal with a data base - or you just like using SQL script.

---

_[View the full topic](https://forum.knime.com/t/duplicate-row-filter/21645)._