Feature request: Remove Empty Rows: Option to remove/retain missing value cells

Hi,

Sometimes the flow produces a table with totally empty rows (has row ID but no columns). Here we cannot use the Empty Table Switch node unless we use the Remove Empty Rows node first. But this node removes missing value cells (if no other value is available in the row) as well which in some cases is undesired.

How about adding an option to choose between removing and retaining missing cells?

:blush:

3 Likes

This is the things that I have been looking for …

image

its really needed:
“adding an option to choose between removing and retaining missing cells”

2 Likes

Hi there @armingrudd,

from node description seems to me that Remove Empty Rows node is for removing all rows that contain only missing values so having option to choose between removing and retaining missing cells would be a bit contradictory. Anyways I add @swebb as I got a feeling he knows more about it :slight_smile:

Br,
Ivan

3 Likes

Indeed, the node is supposed to remove rows that only have missing values i.e. every column in the row must have a missing value.

I’m not sure if @armingrudd is saying it’s removing rows when some columns do have values and some don’t? If so this is a bug, if not then I’m not sure I follow.

Cheers

Sam

2 Likes

As a follow up, I just ran a test locally and the node behaves as I expect it to.

This input:

image

Becomes this output:

image

The rows that have only missing values have been removed. Rows that have at least 1 value are kept. This is the purpose of the node. I’m not sure how a dialog option would make sense here.

Could you provide me an expected input and desired output so I can understand a bit better what you want?

There’s also the Row Splitter (col+) node which can split a table based on missing values.

This config will put rows with a missing value anywhere in the second output:

This config will put rows where all cells are missing to the second output:

Cheers

Sam

2 Likes

I think that @armingrudd means that when he has no columns in the table, he then cannot use the empty row filter on the table to remove all the rows, but even though the table is empty in the sense that it has no cells, it still registers in the Empty Table Switch node as being non-empty because it has rows, which is presumably why he would then like to use the Remove Empty Rows node. My apologies Armin if I’ve got that completely wrong!

Steve

5 Likes

You explained my point perfectly. Thank you @s.roughley .

:blush:

3 Likes

Ah ok.

So if the table has rows but no columns remove all rows. If the row has columns don’t do anything?

2 Likes

The workflow is running automatically. sometimes all columns are filtered. but row IDs remain. So the Empty Table Switch does not work. so I need to remove rows first. But that makes some problems when the table has columns but some rows have missing values which we need to keep for further process. but the Remove Empty Rows node removes them.

I think it means adding an option to choose whether remove missing rows or void rows.

:blush:

2 Likes

Got it so my definition (and thus the node’s) of an empty row is rows that have no data.

Another definition is rows that have no columns. You’d like a checkbox to toggle between those two options on the node.

I think that’s doable.

Cheers

Sam

3 Likes

when the table has columns so there would not be any void rows. there the “remove missing rows” option would work.
and if there is no columns, there would be no missing values and only void rows. there the “remove void rows” would work.

I think we need 2 check box for each. (maybe one needs the void row IDs)

Thank you @swebb

:blush:

2 Likes

Your post was brought to my attention at a lucky time for you.

Unless something goes wrong the node should have the following options within the next hour (nightly update site):

image

So you can now select to remove:

  1. Only rows with no columns
  2. Only rows with entirely empty cells
  3. Rows with either of the above
  4. Neither 1 or 2 - node does nothing

Cheers

Sam

5 Likes

Great job @swebb. Thank you so much.

:blush::+1:

2 Likes

You’re welcome.

It looks like it’s now available in the nightly/trunk build. If you could let me know if this works as expected I’ll then migrate it over to 3.7, 4.0 and 4.1.

Cheers

Sam

2 Likes

I think radio option would be better in this case since both cases would not occur simultaneously.

:blush:

1 Like

Wouldn’t occur on a given input but if you want to remove either instance in a loop a radio button wouldn’t let you do that. For example how the node used to work.

But allowing people to select neither may not be appropriate without a warning so maybe that should be added.

2 Likes

Thanks again for your quick support. I just tested the node and it does the job perfectly.

:blush:

2 Likes

Nice one guys :+1:
Ivan

1 Like

I have merged the changes into 3.7, 4.0 and 4.1. Should be available tomorrow if not builds fail. Available in nightly until then.

Cheers

Sam

2 Likes

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.