# Filter large table **URL:** \ **Category:** KNIME Analytics Platform\ **Created:** [February 27, 2019, 4:46am UTC](https://forum.knime.com/t/filter-large-table/13653 "2019-02-27T04:46:45Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![jeffreysdelong](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/j/c77e96/32.png) [@jeffreysdelong](https://forum.knime.com/u/jeffreysdelong)\ **Post date:** [February 27, 2019, 4:46am UTC](https://forum.knime.com/t/filter-large-table/13653/1 "2019-02-27T04:46:45Z")
I am building a deep learning network (using DL4J nodes) and am reading data from files with over 60K rows. I would like to only include every third row. I.e., I would like to filter out 2 out of every three rows. I transpose the table (so the rows become columns) so I could also filter on the columns if that was easier. Random selection of rows or columns would also be acceptable as long as I wind up with about 20K. I appreciate any suggestions on how to do this. ---
**Author:** ![Andrew\_Steel](https://forum-cdn.knime.com/user_avatar/forum.knime.com/andrew_steel/32/4472_2.png) [@Andrew\_Steel](https://forum.knime.com/u/Andrew_Steel)\ **Post date:** [February 27, 2019, 6:40am UTC](https://forum.knime.com/t/filter-large-table/13653/2 "2019-02-27T06:40:19Z")
Hi @jeffreysdelong, take your table, append a new column with Math Formula and value $$ROWINDEX % 3 and Filter by 0,1 or 2 with a Row Filter. Best regards Andreas ---
**Author:** ![s.roughley](https://forum-cdn.knime.com/user_avatar/forum.knime.com/s.roughley/32/366_2.png) [@s.roughley](https://forum.knime.com/u/s.roughley)\ **Post date:** [February 27, 2019, 6:48am UTC](https://forum.knime.com/t/filter-large-table/13653/3 "2019-02-27T06:48:59Z")
You could also use the `Partitioning ` node if random is ok > **[Partitioning — NodePit](https://nodepit.com/node/org.knime.base.node.preproc.partition.PartitionNodeFactory)** > > Splits table into two partitions. Steve ---
**Author:** ![jeffreysdelong](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/j/c77e96/32.png) [@jeffreysdelong](https://forum.knime.com/u/jeffreysdelong)\ **Post date:** [February 27, 2019, 3:55pm UTC](https://forum.knime.com/t/filter-large-table/13653/4 "2019-02-27T15:55:27Z")
Great suggestions. I will check them both out. Thanks, Jeff ---
**Author:** ![beginner](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/b/e495f1/32.png) [@beginner](https://forum.knime.com/u/beginner)\ **Post date:** [February 28, 2019, 1:25pm UTC](https://forum.knime.com/t/filter-large-table/13653/5 "2019-02-28T13:25:32Z")
Or Row Sampling Node. And the Sampling in these doesn’t need to be random. Just select linear sampling. ![image](https://forum-cdn.knime.com/uploads/default/original/2X/b/b342d3c958282d940dbae7615edbffdf9b359866.png) ---
**Author:** ![s.roughley](https://forum-cdn.knime.com/user_avatar/forum.knime.com/s.roughley/32/366_2.png) [@s.roughley](https://forum.knime.com/u/s.roughley)\ **Post date:** [February 28, 2019, 8:45pm UTC](https://forum.knime.com/t/filter-large-table/13653/6 "2019-02-28T20:45:47Z")
@beginner - thanks - that looks like the Row Filter analogue of the Partitioning node, which also on close inspection has the same options. I guess the final choice depends whether @jeffreysdelong wants the remainder of his dataset as a test set. Steve ---
**Author:** ![jeffreysdelong](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/j/c77e96/32.png) [@jeffreysdelong](https://forum.knime.com/u/jeffreysdelong)\ **Post date:** [March 1, 2019, 4:40am UTC](https://forum.knime.com/t/filter-large-table/13653/7 "2019-03-01T04:40:15Z")
Thanks for all the suggestions. I wound up using Andrew’s suggestion. ![ReduceAndTranspose](https://forum-cdn.knime.com/uploads/default/original/2X/e/e8773d4c4228d65dc8f16590cb0765ad788e60c4.png) The input files have gene expression data in a column. I wanted to reduce the amount of data from 60K to 20K to make it more manageable. I then transpose the column to a row so I can add additional patients data. ---
**Author:** ![beginner](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/b/e495f1/32.png) [@beginner](https://forum.knime.com/u/beginner)\ **Post date:** [March 1, 2019, 8:41am UTC](https://forum.knime.com/t/filter-large-table/13653/8 "2019-03-01T08:41:26Z")
Fine if it works for you but column expressions is very slow. Using partitioning or row sampling will be faster. ---
**Author:** ![system](https://forum-cdn.knime.com/uploads/default/original/1X/ab3ccf34482a0329361734a18199390177204f15.png) [@system](https://forum.knime.com/u/system)\ **Post date:** [June 23, 2023, 8:32pm UTC](https://forum.knime.com/t/filter-large-table/13653/9 "2023-06-23T20:32:48Z")