Weird behavior in Column Expression node with arrayReverse function

[justin.m]

I think I’ve found a bug in the arrayReverse function in the Column Expression node, and I just wanted to check I wasn’t missing something.

It looks like when you use the function on a column containing a list, it gives you the same list in reverse order. But, if you reference the same column in a later expression, you get the reversed version. This does not occur when you click the ‘Evaluate’ button, only when you run the node.

I’m attaching a simple workflow that shows what I mean. It starts with single column called ‘Values’, containing the list [1,2,3,4,5]. This feeds into a column expression node containing 4 expressions, all collections:

• Values 1:
  • The expression is ‘column(“Values”)’.
  • Clicking ‘Evaluate’ gives [1,2,3,4,5].
  • When run, it outputs [1,2,3,4,5].
• Values Reversed 1:
  • The expression is ‘arrayReverse(column(“Values”))’.
  • Clicking ‘Evaluate’ gives [5,4,3,2,1].
  • When run, it outputs [5,4,3,2,1].
• Values 2:
  • The expression ‘column(“Values”)’.
  • Clicking ‘Evaluate’ gives [1,2,3,4,5].
  • When run, it outputs [5,4,3,2,1].
• Values Reversed 2.
  • The expression is ‘arrayReverse(column(“Values”))’.
  • Clicking ‘Evaluate’ gives [5,4,3,2,1].
  • When run, it outputs [1,2,3,4,5].

Even though the last two expressions are exactly the same as the first two, they give different results.

arrayReverseStrangeness.knwf (8.4 KB)

[dora_gcs]

Hello @justin.m ,

thank you for your detailed description and also for the sample workflow.

What you see is intended. The reverse() method reverses the order of the elements in an array AND it overwrites the original array.

You can see it in the description of the method, inside the node:

What could be misleading is that clicking the ‘Evaluate’ buttons triggers evaluation of the current expression regardless if the input array was overwritten or not by an earlier function.

I hope I could help!
Dora

[justin.m]

Thanks @dora_gcs ,
I will admit I hadn’t noticed that line in the function help, thanks for pointing it out. I’m not sure it conveys the scope very well. For instance, if you use arrayReverse, it affects all later expressions, but the column is not changed in the output.

However, I still think there is something funny happening. I’m attaching an updated version of my previous workflow, with an additional branch. The first column expression node in this branch has two expressions:

• The first adds a “6” to the original array. This replaces the original column.
  *The second expression reverses the original array and adds the results as a new column.

There is another column expression node that converts them both to strings.

When I run this branch, half the time it results in:
|Values | Values Reversed 1|
|{1,2,3,4,5,6}|{5,4,3,2,1} |

and the other half gives:
|Values | Values Reversed 1|
|{5,4,3,2,1,6}|{5,4,3,2,1} |

Note the first column results are different. So the node is producing different results, with no change in the input or node settings. It seems like the results of the first expression are being affected by the second expression, some of the time.
arrayReverseStrangeness2.knwf (12.3 KB)

[dora_gcs]

Hello @justin.m ,

the behavior what you described is caused by the fact that the Column Expressions Node currently doesn’t allow to use new output columns created in expressions to be used in the following expression (as an input data) of the same node. In your case you added a string to your list and wanted to replace that column and later on you wanted to use the newly replaced column in the second expression in the same node. But it is not possible currently.
We have a feature request to make this possible (internal reference number is AP-13383) which has no ETA at the moment.

As a workaround you can use a second node for using the newly created (‘replaced’) column as an input for your next expression.

I hope this helps!
Dora

[justin.m]

Hi @dora_gcs ,
That wasn’t the bit that concerned me. I don’t expect the second expression to use the replaced column value. The weird behavior is that the replaced column is producing different results, when given the same input and the same configuration.

I just tried using a counting loop to run the same node 20 times and this is the output:

Values	Values Reversed
{1,2,3,4,5,6}	{5,4,3,2,1}
{5,4,3,2,1,6}	{5,4,3,2,1}
{5,4,3,2,1,6}	{5,4,3,2,1}
{1,2,3,4,5,6}	{5,4,3,2,1}
{5,4,3,2,1,6}	{5,4,3,2,1}
{5,4,3,2,1,6}	{5,4,3,2,1}
{1,2,3,4,5,6}	{5,4,3,2,1}
{5,4,3,2,1,6}	{5,4,3,2,1}
{5,4,3,2,1,6}	{5,4,3,2,1}
{1,2,3,4,5,6}	{5,4,3,2,1}
{5,4,3,2,1,6}	{5,4,3,2,1}
{5,4,3,2,1,6}	{5,4,3,2,1}
{1,2,3,4,5,6}	{5,4,3,2,1}
{1,2,3,4,5,6}	{5,4,3,2,1}
{5,4,3,2,1,6}	{5,4,3,2,1}
{1,2,3,4,5,6}	{5,4,3,2,1}
{1,2,3,4,5,6}	{5,4,3,2,1}
{1,2,3,4,5,6}	{5,4,3,2,1}
{5,4,3,2,1,6}	{5,4,3,2,1}
{5,4,3,2,1,6}	{5,4,3,2,1}

If you look down the first column, sometime it starts with a 5, sometimes with a 1. So the arrayReverse function is sometime affecting the previous expression, and sometimes it isn’t.

[dora_gcs]

Hello @justin.m ,

thank you for your patience in this topic!

We have discussed this behavior internally and our developers opened a development ticket for this bug. Internal reference is [AP-20461] which will be fixed hopefully in the next release (4.7.3), coming in the next couple of weeks. You can follow up this fix in our changelog. Thanks for your contribution!

Kind regards,
Dora

[justin.m]

Thanks @dora_gcs , I appreciate your help.

[forum]

Internal ticket ID: AP-20461
Fix version(s): 5.1.0, 5.0.1, 4.7.3, 4.6.6
Other related topic(s): -