Linked-shared notes within linked-shared notes does not work.

Hi community,
I tried to share a simple component and link it. But when I share and link another component, I cannot use it within the first component.

e.g. recreate this:

  1. create a workflow
  2. create two components (no matter where)
  3. share and link both components (no matter how)
    4 drag&drop one of the components into the other component

Now try either of the following:

  • within the first component, right-click the second component which you pulled into the first component. >> No menu is opening.
  • even if a menu is opening after 10-30 trails, there will be no menu for the component available, see picture below, linked green-arrow component without component menu.
  • save and re-open the first component, which holds the second component. It will always run into error, failing to update the integrated component as it cannot be found.

image

Question 1: Is that purpose or bug? No linked component within a linked component?

Questions 2: Is there a best practice for the following use case?

I call multiple times a database session, within a moderately complex workflow. Instead multiple times actually putting a copy of the note which calls the data base session, I want that within a lined component, so I can adjust PW and CREADTIALS if needed at one ‘setting’ place, but not at multiple places. I also do not think it is advisable to push the CREADENTIALS and PW via variables through the whole workflow.
This setting component is actually easy and works.

HOWEVER, THIS WOULD BE MY SECOND COMPONENT (from the example above), WHICH I WOULD LIKE TO INTEGRATE IN ANOTHER COMPONENT, WHICH FOR SIMILAR BUT OTHER REASONS I WANT TO LINK AS WELL.
But here I run into the error.

I hope I could explain well. If not, I am happy to specify.

Thanks
D

1 Like

Hi @d00m , your case is a good one to use as a component indeed. And sure you can use a component within a component.

For reusable components, you can check out my explanation in this thread:

1 Like

Hi @bruno29a,
Thank you very much for the explanation, this kind of library is, what I have in mind for me and my organization as well.

But still, the issue is this:
Yes, I was able to use a component within a component. However only, if the containing component is not shared. Once you share the containing component, the issue occurs.

In pictures:
The original workflow contains ‘Component’

‘Component’ is shared and contains ‘Component 2’

‘Component 2’ is shared and has something to do.

Now, within ‘Component’, I cannot update ‘Component 2’, once ‘Component’ is getting shared.

Question: Is this a bug on my side, a bug on KNIME side in general, or on purpose for some good reason?

Thanks for clarification
D

As far as I know a linked component can only be updated at the “source”. So in the component 2 itself. Otherwise you have to unlink it. So I would not say it’s a bug.
br

Hi @d00m , so, just like when you include a shared linked Component in a workflow, where you cannot modify that Component, it works the same way when you include it in another Component. You cannot modify a shared linked Component. You can only modify the source, and then re-save the Component.

It’s one of the pain that I had to deal with when using Components within Components, but on the other hand, it’s also a sort of security, reminding you that you are trying to modify a shared linked Component that will impact other workflows that are using that Component.

I try to avoid using Components within Components because of that, because I’d have to update 2 sources if I’m updating a Component used within a Component. Only in rare cases where I’m creating a wrapper Component that I do Components within Components. But I try to avoid it where possible.

For example, let’s say I can fetch articles from a data source only based on a date range. Everything else has to be filtered after the data is retrieved. I can have a Core Component called “Fetch All Articles” where the data is retrieved and common logic is applied, after which I may have wrapper Components such as “Fetch Articles By ArticleID”, and “Fetch this by AuthorID” where they would call the Core Component “Fetch All Articles” and apply filter by ArticleID for the “Fetch Articles By ArticleID” Component, and apply filter by AuthorID for the “Fetch this by AuthorID” Component.

Of course, if the user is going to do both kinds of fetch, then you don’t want to be running both wrappers, as they would both call the Core Component and retrieve the same data twice from the data source, in which case it’s better that the Core Component “Fetch All Articles” is used, along with manual filtering of AuthorID and ArticleID.

Alternatively in this case, I can also build the 2 Components “Fetch Articles By ArticleID” and “Fetch this by AuthorID” without including the Core Component in them, but rather as Components that assume that the Core Component is called before calling them, and therefore can be built based on the expected results from the Core Component, hence avoiding to include a Component within a Component.

The other issue I don’t like about shared Linked Components being used within other shared Link Components is if you need to somehow share your workflow to someone who does not have these Components or does not have the same workspace structure, Knime will complain that it cannot find the Linked Components, and it’s a pain to have to unlink these Components, especially those inside the Components.

But you get the idea for the use of Components within Components, the pros and cons.

You are right, and this makes sense, I should only be able to change such functions/processes/components within the source.
I was misleading with my wording.
I do not mean “Update the source component itself”
I mean “Update the link” or “Update the instance of the component”, e.g. unlink the instance, or avoiding the error I constantly got, saying: cannot update the component, because it cannot find it.
The error disappeared for some unknow reason, but the link-update is not possible still.

I can do this within an unlinked workflow / component:
image

I cannot to this within a linked component:

I do not see a reason, why these both cases should be different.

…just like when you include a shared linked Component in a workflow, where you cannot modify that Component, it works the same way when you include it in another Component. You cannot modify a shared linked Component. You can only modify the source, and then re-save the Component.
D: I understand that, this makes sense, no issue here.

It’s one of the pain that I had to deal with when using Components within Components, but on the other hand, it’s also a sort of security, reminding you that you are trying to modify a shared linked Component that will impact other workflows that are using that Component.
D: I understand that, this makes sense, no issue here.

I try to avoid using Components within Components because of that, because I’d have to update 2 sources if I’m updating a Component used within a Component. Only in rare cases where I’m creating a wrapper Component that I do Components within Components. But I try to avoid it where possible.
D: I understand that, this makes sense, however it would be nice if KNIME had a routine/process/what so ever, that allowed to update the chain of components, that are identified: e.g. A workflow opens > asks for updating included linked components > identifies 2nd layer components, that are within those 1st layer components, in order to update the first layer component properly > DOES THE SAME thing as long as there are layers of linked components. KNIME could do this even in the background and prompt a result to the user what’s the situation and what the user wants to do about it, e.g. update all automatically, or let the user select, which to update and which to unlink, if allowed to.

For example, let’s say I can fetch articles from a data source only based on a date range. Everything else has to be filtered after the data is retrieved. I can have a Core Component called “Fetch All Articles” where the data is retrieved and common logic is applied, after which I may have wrapper Components such as “Fetch Articles By ArticleID”, and “Fetch this by AuthorID” where they would call the Core Component “Fetch All Articles” and apply filter by ArticleID for the “Fetch Articles By ArticleID” Component, and apply filter by AuthorID for the “Fetch this by AuthorID” Component.
Of course, if the user is going to do both kinds of fetch, then you don’t want to be running both wrappers, as they would both call the Core Component and retrieve the same data twice from the data source, in which case it’s better that the Core Component “Fetch All Articles” is used, along with manual filtering of AuthorID and ArticleID.
D: You are right, in this example, if “Fetch Articles By ArticleID” and “Fetch this by AuthorID” are close to each other within the visual interface, than it makes little sense, to call the same core process twice in advance e.g. “Fetch All Articles”. However, if both wrapper components are far from each other, or within different workflows but within components, then what would be the best practice, to carry the results from the core component towards the point, where it is needed, even if it is needed more than one or two times? In my original case, it is about a core component, that opens a data base session, and I need this session multiple times in different places of the rather complex workflow (maybe in the case of database session, there are other elegant solutions, which I am happy to read about). But another, common use case is e.g. a “DateManipulation”-linked-component, which has a simple user interface (see picture below). This can be used multiple times with different settings via the interface, and at best even within other LINKED components e.g. a ‘DataBaseSetting’ with a similar interface, setting the database credentials or other needed things. Both could be standardized and intensely used by me or others in multiple projects, and therefor at best linked.

Alternatively in this case, I can also build the 2 Components “Fetch Articles By ArticleID” and “Fetch this by AuthorID” without including the Core Component in them, but rather as Components that assume that the Core Component is called before calling them, and therefore can be built based on the expected results from the Core Component, hence avoiding to include a Component within a Component.
D: but see above, what if the wrapper components are far from each other?

The other issue I don’t like about shared Linked Components being used within other shared Link Components is if you need to somehow share your workflow to someone who does not have these Components or does not have the same workspace structure, Knime will complain that it cannot find the Linked Components, and it’s a pain to have to unlink these Components, especially those inside the Components.
D: would be easily avoided, if KNIME had a prompt when opening the workflow: "Those are the components within that workflow. This is the sub-list, which cannot be updated. What do you want to do? Find and relink them? Unlink them (all or a selection via check box)?"

But you get the idea for the use of Components within Components, the pros and cons.
D: yes, I get your point, but it is either a pity that it is not easily possible, what is possible in other languages: define a function/procedure/etc. and reuse it, where ever needed, even within myself.

I hope I do not confuse to much, and I am happy about your support.

2 Likes

Hello @d00m,

this discussion seems related? (And possibly offers solution?)

Br,
Ivan

Hi @d00m , no, it’s not confusing at all, don’t worry. Ok, I get what you are saying, and also definitely some interesting features that you brought up to give more control to the user, especially where you can choose what to do per component. This could be an issue though if the workflow is run as a job by the system where there is no such interaction. But there probably could be some default behaviour that can be set up that the system would use.

For " what if the wrapper components are far from each other?", it should not be an issue, as long as the core was executed before the wrapper, no matter where. Obviously it’s not the ideal structure, but I would not count it as an issue.

“a pity that it is not easily possible, what is possible in other languages: define a function/procedure/etc. and reuse it, where ever needed”. Yes indeed, but I think Knime was not meant to provide this kind of possibilities and structure. At least Components provide something a bit close to that. But hopefully they can be improved, and feedback/ideas/discussions such as provided here are always helpful to help for improvement.

Hopefully the Knime team is taking notes :smiley:

2 Likes

Thank you @ipazin
I think I encountered the same as dobo. And I see the issue is realized.
Best
D

1 Like

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.