value = int(value) test for Rule Engine

I want to use a rule engine node to tag rows where the double value >2 and <8 and the value is an integer. Is this possible?

Thanks

tC/.

As far as I’m aware, there is no single function that will do this and you’ll have to string together existing functions.

One approach would be to use the Column Expressions node with the expression:

and(between(2,column("your column name") ,8 ), equals(floor(column("your column name")),column("your column name")))

This expression returns “true” if the value in your specified column is between 2 and 8 AND if the value in the column is equal to the largest integer below it:

image

2 Likes

Sure - use an expression like this:

($column1$ > 2 AND $column1$ < 8) => 1
TRUE => 0

2020-10-19 13_53_13-Classified values - 0_399 - Rule Engine

Hi @ScottF … I got that far, it is when I needed to add the 2nd part to ‘rule’… AND $column1$ = int($column1$) that I came unstuck. Using @elsamuel 's solution does the job with a column expression node with a few extra.

This is flow is working through 70 million rows so every node is precious, that is why I was hoping to get a single rule engine node to do all the work.

tC/.

Ah, I missed that you were needing a Double to Integer type conversion. In that case you could use a Double To Int node after the Rule Engine. You might actually want to compare the timing for using these two nodes versus a single Column Expressions node using a Timer Info node - it’s possible the two node approach is faster in this case.

Also, in general, you may want to consider enabling streaming to increase performance here.

1 Like

I am loving streaming … I have been using as much as I can with this project. So far I have cut about 3 hours out of processing time.

2 Likes

Hello @TigerCole,

using @elsamuel’s approach (nice one :+1:) you can accomplish same in Math Formula node as well:
if( between(2, $column1$, 8) && floor($column1$) == $column1$ ,1 ,0 )

Br,
Ivan

1 Like

Hi @ipazin and @elsamuel

Nice to know that I can use the same logic in the math node.

I have put the “@elsamuel solution” to work and it has proven to be a very robust and elegant solution. When the column expression node is followed with a rule engine node the results are incredible. I have an expression node that has 15 expressions testing and appending a column with a true/false result and then I use the rule engine node to tidy up.

For example, I am segmenting travel behaviour into a number of windows based on time of day and day of the week and distance travelled. Based on the pattern of the true/false output I can assign any of the 70million rows to any of the “traveller” classes with the rule engine.

The unexpected advantage is the flexibility of the solution, I can add or update test expressions easily to refine the outputs and then make any changes I need to the rules in the rule engine. This is proving to be really valuable simplicity for the researchers using the data.

…br

tC/.

3 Likes

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.