Filter data for specific column and add remarks for blanks

Hello,

Here’s my sample dataset.

TYPE Jul Aug Sep Jul (#1) Aug (#1) Sep (#1) Jul (#2) Aug (#2) Sep (#2) Jul (#3) Aug (#3) Sep (#3)
ABC T1 - Tagging T2 - Tagging T1 - Tagging Vol T2 - Tagging Vol
ABC
ABC 0 0 210 0 0 214 0 0 87570 0 0 188700
ABC 0 0 783 0 0 55 0 0 83400 0 0 130000
ABC 0 0 783 0 0 55 0 0 78300 0 0 130000
ABC 0 0 130 0 0 55 0 0 316300 0 0 130000
ABC 0 0 30 0 0 55 0 0 110900 0 0 188700
ABC 0 0 3 0 0 4 0 0 11500 0 0 188700
ABC 0 0 783 0 0 845 0 0 326511 0 0 188700

I have only two expected output:

  1. How can I copy the remarks for example under Jul column, copy the “T1 - Tagging” to Aug and Sep. Then for Jul (#1) to Aug (#1) and Sep (#1), so on and so forth.

  1. May I ask how can I filter a specific column or months, for example, this month is Sep, so it will only filter the months for Sep, Sep (#1), etc. Then next month, for example the month will be, December, then it will get Dec, Dec (#1), etc. Is it possible to make it as variable?
TYPE Sep Sep (#1) Sep (#2) Sep (#3)
ABC T1 - Tagging T2 - Tagging T1 - Tagging Vol T2 - Tagging Vol
ABC
ABC 210 214 87570 188700
ABC 783 55 83400 130000
ABC 783 55 78300 130000
ABC 130 55 316300 130000
ABC 30 55 110900 188700
ABC 3 4 11500 188700
ABC 783 845 326511 188700
  1. Similar case to item #2, it is just that what if I want to get 2 months which is current and prior month. Assuming today’s month is September, so it will get Sep and Aug.
TYPE Aug Sep Aug (#1) Sep (#1) Aug (#2) Sep (#2) Aug (#3) Sep (#3)
ABC T1 - Tagging T1 - Tagging T2 - Tagging T2 - Tagging T1 - Tagging Vol T1 - Tagging Vol T2 - Tagging Vol T2 - Tagging Vol
ABC
ABC 0 210 0 214 0 87570 0 188700
ABC 0 783 0 55 0 83400 0 130000
ABC 0 783 0 55 0 78300 0 130000
ABC 0 130 0 55 0 316300 0 130000
ABC 0 30 0 55 0 110900 0 188700
ABC 0 3 0 4 0 11500 0 188700
ABC 0 783 0 845 0 326511 0 188700

Hope you can help me with this. Thank you so much!

Hello, hope someone can help if this is possible. Thank you!

@takbb

Hi @trafalgarlaw , I am not sure if I’ve fully understood your requirement, but give this a go

Everything up to the concatenate node is for copying across the remarks to populate the blank columns to their right. It doesn’t have any logic to determine where it is copied, other than it just fills where they are missing. To do that it transposes the data in the first row and uses Missing Value to “fill from previous”, and then transposes back again.

Parts 2 and 3 are covered by the component at the end. I did this as a component so that you could configure it. Of course you could do it without being a component use Variable Creator instead to set the required variables.

Within the component it filters to the rows containing the requested Month name(s).

image

I added a third option which is to include the months either side of the chosen month. I did this just to show how it could be extended further if needed.

Filter data for column and replace blanks.knwf (147.6 KB)

3 Likes

Thank you so much, @takbb! Will check on this and get back to you once all good :slight_smile:

Hello @takbb, in the attached workflow, I can’t see the below.

I can only see this:

If you don’t mind, can I request a workflow which is not a reset workflow so I can see the output on my end? Thank you!

Hi @traflgarlaw, there should not be anything in the Workflow preventing it executing. It is self contained. The upper screenshot is what you see if you select the component on the far right, and right click on it “open component”.

Let me know if you cannot execute it, and what happens when you try? Which version of KNIME are you using ?

1 Like

Hello @takbb, saw why it can’t be executed, because of Row Splitter. I am in 4.7.8 KNIME Version.

Would you mind sharing your configuration of row splitter so I can use the row splitter available with my version. Thank you so much!

Tried zooming it out and checking the component part, but still not part of the workflow you’ve attached. See below workflow editor:

Ah OK. The Row Splitter has been updated to Modern UI in the latest KNIME versions (e.g. 5.3). It can be frustrating that this means in earlier KNIME versions it isn’t found, even though there is nothing special about the config. Anyway, it’s simply set to filter Row 1 to the top port and all other rows to the bottom port.

If you still have problems, let me know and I’ll port it to 4.7.8 when I have a chance. It’s probably worth always stating on your questions if you are restricted to using an older version of KNIME as it can affect how a solution is approached, and what nodes are compatible, and I will always assume people will be on the latest (or recent) version unless they specifically say otherwise.

Although I can understand either wanting, and in many cases needing, to stay on 4.x, and I know 4.7.8 was released (just) less than a year ago, it is becoming increasingly difficult for people to support on the forum, as it means having to keep an old copy installed, and 5.x is beginning to offer many new nodes which make it less and less compatible with 4.x.

On your other note, the component you want to open to see the workflow detail I posted is the one at the end, where it says “Choose Month Range”. You may find problems with that in 4.7.8 because a number of nodes it uses now have “Modern UI” configuration too.

image

I opened in 4.7.8 and edited the new nodes and resaved it for you.

Filter data for column and replace blanks v478.knwf (202.8 KB)


A problem with the refresh of the KNIME nodes is that it is impossible to tell which of the new-look (Modern UI) nodes will work in an older version and which won’t.

For example, Column Filter, Table Row to Variable, and Column Combiner have all been updated to the Modern UI config, but they work fine when opened in 4.7.8. Whereas other nodes (Row Filter and Row Splitter) have had other functional changes and are no longer seen as the same node in 4.7.8, so they appear “missing”.

btw, you should have got a warning when you opened the 5.x workflow in 4.7.8, to tell you it might not be compatible. Did that not happen, as that should have been a clue why the workflow wasn’t working, rather than asking me to upload it without resetting, which wouldn’t have helped in this case anyway.

2 Likes

Thank you, @takbb! Saw it clearer as water now! :slight_smile: Very expert and reliable! Love your welcoming and approachable nature as always!!!

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.