Select columns of table at random

[mgamer]

Related: click
I have a table consisting of about 10k Rows and 250 columns, now I want to choose a number of, say 100, columns at random from this table and can not manage this up to now. I made 100 random numbers with the node Random Data Generator, transposed them and then tried to select the columns with node Reference Column Filter. But I could not get this to work. Can anyone give me a hint?

Greetings from Hanau / Germany

Mike

[HansS]

Hi @mgamer Mike

Maybe this wf random_column_select.knwf (84.3 KB) helps you along with your challenge. I think you where almost there. After the Transpose node, I suggest a Row Sampling node and then Transpose the data “back” again. I added a Column Splitter node in case you don’t want to sample all of them.

gr. Hans

[mgamer]

Hi @HansS Hans,
thank you for the fast answer. This works, but I get the problem that the second transpose operation (in my case a 100x10000 Matrix lasts a few minutes (!). The first transpose with a 10000x250 matrix is done in two or three seconds. But I can use this for the time until I get a faster solution.

[izaychik63]

You may try KNIME component.

Column Filter (by Index) – knime

Generate 100 random nubers;
Convert them to string;
Use group by to make a parameter string and pass it to component.

[HansS]

Hi @mgamer Mike,

I made a small adjustment to the wf random_column_select.knwf (89.9 KB) . After the Column Splitter I created a Row Filter node, to filter out just 1 row. So both Transpose nodes should go much faster and is input for the Reference Column Filter.
gr. Hans

[mgamer]

Hi @izaychik63,
thank yo for this solution - cool

[mgamer]

Hi @HansS this is much faster now - thank you very much. I’m doing most of my work in Mathematica, so I’m not very familiar with KNIME and use it most often to validate the Mathematica results (just for sure…)

Greets from Hanau/Steinheim
Mike