Python- How to loop over columns in using Python integration

KNIME Analytics Platform
1

Hello,

I know how to iterate over rows in a python script, but I would like to iterate addtionally over columns. Is there a syntax for this?

Thank you,

Marc

 

2

Hi Marc,

if I understand you correctly then this is what you are looking for:

for column_name in input_table:
    column = input_table[column_name]

Cheers,

Patrick

3

Hi Parick,

thanks, I actually just figured it out myself. Sometime I make my life unnecessarily complicated...

Marc

 

 

1 Like
4

Hi Patrick,

I need to loop over the columns and compare the columns and if the result is positive, I have to create a new column with the right id.

Till now I´m using 6 rule engine nodes, but I believe I can improve my workflow.

1º rule

$ColumnValues$ = $id$ AND $Posicao$ <=4 AND $Posicao(-1)$ < 1 => $ColumnValues$

output column = FT

2º rule

$Posicao$ =5 AND $Posicao(-1)$ < 5 => $ColumnValues$

output column = TL

3º rule

$Posicao$ =6 AND $Posicao(-1)$ < 6 => $ColumnValues$

output colum = M

and so on…

image

image
image1079×110 3.18 KB

5

Hi there @wpaiva,

I guess you would like to do this in Python and you encountered a problem or?

Additionally you can replace all 6 Rule Engine nodes with one Column Expressions node :wink:

Br,
Ivan

1 Like
6

Column Expressions node however is extremely slow (unless with KNIME 4 that has been improved a lot).

2 Likes
7

Hi there @beginner ,

Although it is slower compared to Rule Engine node I haven’t experienced extreme slowness with Column Expressions node :slight_smile: On the other side it does offers calculations and possibility to implement more complex logic.

Did you maybe test speed execution it and can share your result to check it out?

Br,
Ivan

1 Like
8

I tested the node back when it released and yeah what it offers is great but the speed trade-off I had in that specific use-case which I don’t remember was not worth it to me. I’m also not talking about that many rows, 100ks to millions. Still being 10x faster will be noticeable with 1 mio rows.

2 Likes
9

Thank you all!

I´m using the Rule engine node perfectly.

1 Like
10

Hello!

Just to drop info that Column Expressions got speedup with 4.2.1. See here: Column Expression Speedup with 4.2.1

Br,
Ivan