Removing blank cells individually, not entire rows/columns

Hello!

In my current workflow, I remove the outliers of measurements. This leaves me with several blank cells that I want to remove in order to clean up my data.

I have attached the KNIME export document, and also an example of what I am hoping to achieve.

I have been searching through the forums for a couple hours and have only been able to find information about filtering out entire columns or rows that contain missing/null cells.

Thanks in advance!
Eccentricity_Fixed.xlsx (19.5 KB)
Eccentricity.xlsx (21.1 KB)

1 Like

Hi @nagemneuman,

I built little workflow that transforms your original table into the fixed table.

The idea is to use a loop that processes one row after the other. In each iteration the workflow

  • removes all columns with missing values
  • calculates the mean of all remaining columns and
  • performs some column reordering and renaming.

Please let me know in case of any questions.

Cheers
Kathrin

6 Likes

@Kathrin 's option is certainly a good solution in KNIME. Other option would be to use a python snippet with numpy.
br

Thanks so much Kathrin, worked like a charm!

The only thing that’s off is that when I put it into my workflow, I’m missing some of the CellBody counts. No big deal because I can just add them in manually, but I’ll attach a screenshot. It should read as 0 1 2 3 4 5.
Example

hm that is strange. If I execute the workflow that I prepared for you the CellBody counts has the correct values.

Do you have this issue if you execute the workflow I shared with you?

1 Like

No I don’t, but I copied and pasted your workflow right into mine… The only difference was that I couldn’t make the CellBody an IntValue when renaming. Does that have anything to do with it?

I changed the data type of the column CellBody to string and it still works as expected. So I assume that this is not the issue.

Can you please check which node actually loses the CellBody information, by checking the output of each node?

1 Like

Okay, I didn’t know I could check the output of each node until you said something. It’s a node upstream of the workflow you provided. So thank you, and sorry!

excellent! Happy that my workflow works solves your problem :slight_smile:

Okay so I’m back with another question!

This is a sample of the workflow I’m working on (see Sample Workflow). And I can get it to work the way you helped me to when it analyzes 1-2 image sets (see 2 Image Sets for original data). But when I have a document containing 3 or more image sets (see 4 Image sets for original data), it doesn’t work properly (see Eccentricity for export)?

2 Image Sets.xlsx (726.7 KB)
4 Image Sets.xlsx (1.4 MB)
Sample Workflow.knwf (67.7 KB)
Eccentricity.xlsx (39.9 KB)

Hi @nagemneuman,

if I understand the issue correctly it seems like the last column resorter node causes the issue.

Can you please check the output of the last Math Formula node in your workflow. Does this table looks correct to you in case of 4 images?

If yes, please create a new column resorter node and make sure

  • to only move the column Cov and
  • that the entry is at the bottom.

Cheers
Kathrin

Worked like a charm! Thank you so much for your help!!

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.