Bug using Loop End (Column Append)

Hi,

I am undertaking the following operation on a table of 133 columns and 1300 rows. The task being undertaken is taking a selection of these columns (aroung 60), then extracting the column header, transposing this, and using this in TableRow to Variable Loop Start.  The data just before the column header extraction node is fed into this loop just after the TableRow to Variable Loop Start node. The first node within the loop is a Column Filter node which uses the variable (generated from the column headers) such that only one column is passed through. There are then two String Replacer nodes, which replace all occurances of a "<" with nothing and a ">" with nothing. Then follows "Loop End (Column Append)".

The bottom line of what its doing is using Column filter so it takes one Column at a time, and replacing strings < and > with nothing within the column. These are then all combined together at the Loop End (Column Append).

This loop takes around 1 minute to complete, however, if you reset this loop and run it again, and do this about 10 times, then KNIME comes to a complete halt and very slow. The heap status is still very low, so I am unsure what is causing this.

Simon.

Dear Simon,

unfortunately I can't help you with your problem (I am a newbie...).

I have been working on creating a similar workflow you describe (column by column selection from a given table in a loop), but can't get it to actually run: the column-selection node complains: "Error loading flow variables into node: Can't parse selection option: CORE" (CORE is the name of my first column). It would be great if you could provide me with screenshots of the settings in your column-selection node (christian.pilger@basf.com).

Regards, Christian

Yes, this was a bit fiddly to set up actually.

In the main config window of Column Filter, move a single column into the include box, any column (dont worry it wont stay there once the variable is set, but you do need to select one), and select Enforce Inclusion. Now goto the flow variables tab and under exclude (I believe the name exclude is a bug! it should be include!), there is a header which says array-size and then under this it says 0, select the variable in this dropdown next to 0. That should do the trick.

Let me know if the problem continues. This flow variable selection in the Column Filter node certainly needs tidying up a bit by the KNIME guys.

 

Simon.

I guess I would hardly have found this by myself - you saved me days of frustration ;-)

Here is definitly soemthing that needs to be fixed.

Thank you, Christian