Advanced FLowvariable Usage in Pivoting Node

Hi,
I’m trying to use Pivot Node with flow variable customization.
According to previous variable I’ve to aggregate data in different ways so I would use flow varialbes. I’ve read the topic [solved] Using flow variable in Pivoting node - #8 by ipazin but it’s not good for my cause.
In that topic the solution is the creation of aggregation variables by standard interface and the override of them by flow variables.

In my scenario, the number ov aggregaion variables is … variable so I cannot populate agggregation column interface.

When I opened PivotNode flow variable interface I’ve found the aggregation column section
In this section I can see 4 variables:

knime_1

So I created 4 string array variables (according to types listed in image). Then I passed them to node.

knime_2

The problem is that, with this configuration, the node display a configuration error with this stacktrace (exracted by a verbose logging).

> 2020-04-24 16:58:06,090 : DEBUG : KNIME-Worker-45-Table Row to Variable 0:32 :  : Node : Pivoting : 0:34 : Errors loading flow variables into node : String for key "numericColumnMethod" not found.
> org.knime.core.node.InvalidSettingsException: Errors loading flow variables into node : String for key "numericColumnMethod" not found.
> 	at org.knime.core.node.workflow.SingleNodeContainer.applySettingsUsingFlowObjectStack(SingleNodeContainer.java:355)
> 	at org.knime.core.node.workflow.SingleNodeContainer.access$2(SingleNodeContainer.java:328)
> 	at org.knime.core.node.workflow.SingleNodeContainer$1.preConfigure(SingleNodeContainer.java:283)
> 	at org.knime.core.node.Node.configure(Node.java:1836)
> 	at org.knime.core.node.workflow.NativeNodeContainer.performConfigure(NativeNodeContainer.java:528)
> 	at org.knime.core.node.workflow.SingleNodeContainer.callNodeConfigure(SingleNodeContainer.java:304)
> 	at org.knime.core.node.workflow.SingleNodeContainer.configure(SingleNodeContainer.java:196)
> 	at org.knime.core.node.workflow.WorkflowManager.configureSingleNodeContainer(WorkflowManager.java:5996)
> 	at org.knime.core.node.workflow.WorkflowManager.configureNodeAndPortSuccessors(WorkflowManager.java:6155)
> 	at org.knime.core.node.workflow.WorkflowManager.doAfterExecution(WorkflowManager.java:3198)
> 	at org.knime.core.node.workflow.NodeContainer.notifyParentExecuteFinished(NodeContainer.java:630)
> 	at org.knime.core.node.workflow.NodeExecutionJob.internalRun(NodeExecutionJob.java:248)
> 	at org.knime.core.node.workflow.NodeExecutionJob.run(NodeExecutionJob.java:124)
> 	at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:334)
> 	at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:210)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> 	at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:123)
> 	at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:246)
> Caused by: org.knime.core.node.InvalidSettingsException: String for key "numericColumnMethod" not found.
> 	at org.knime.core.node.config.base.ConfigBase.getString(ConfigBase.java:405)
> 	at org.knime.base.node.preproc.groupby.GroupByNodeModel.loadValidatedSettingsFrom(GroupByNodeModel.java:485)
> 	at org.knime.base.node.preproc.pivot.Pivot2NodeModel.loadValidatedSettingsFrom(Pivot2NodeModel.java:869)
> 	at org.knime.core.node.NodeModel.loadSettingsFrom(NodeModel.java:462)
> 	at org.knime.core.node.Node.loadModelSettingsFrom(Node.java:641)
> 	at org.knime.core.node.workflow.NativeNodeContainer.performLoadModelSettingsFrom(NativeNodeContainer.java:534)
> 	at org.knime.core.node.workflow.SingleNodeContainer.applySettingsUsingFlowObjectStack(SingleNodeContainer.java:353)
> 	... 18 more

Can you help me?
Thanks

Hi pigreco,

which Version of KNIME are you using?
I think the problem is that one column you are using is not supported by the assigned aggregation method
Or the column name you are giving in the field columnNames does not match the given type
->do you need to put in the columnTypes?
…However this is justed guessed from the error message :grin:

I tried to somewhat do the same as you (in my Version it does not seem like I can give the columnTypes via. array -> therefore the question :slight_smile: )
flow_pivot_example .knwf.knwf (16.5 KB)

Could you maybe give your workflow as an example without real data? Maybe that makes it easier to check what is wrong

Hi,
the version is not the problem. (I use last 4.1.2)
If you try to open flow variables tab after you add aggregation column, you cannot pass columntype as array (you’re forced to enumerate all column types).
But if you remove all aggregation columns in tab "manual aggreagation " and then re open flow variables tab, now you can pass all aggregation params as arrays.
Try it if you can.

In the meantime I’ll prepare a sample workflow.

ah okay :+1:
yes - will try to help you based on your example!
Does not have to be anything major - just something where i can get the error by myself :slight_smile:

Hi @pigreco

I was reading through this, and even so it is possible would not recommend using the flow variables for this case. Maybe we can find another solution together.

What are you trying to achieve? How is the mapping from Column Names to Aggregation done?

1 Like

Hi Iris,
I want create a component usable to process a lot of data organized by date.

For example, COVID data:
so in the first case I’ve for every cities three variables
in second, for regions I’ve 6 variables.

The goal is display data in a line chart but, in the first case I’ve to plot 3 lines but in the second 6 variables.
My component is parametrized by file (or directory name) and an array of column names.

So I’ve to parametrize Pivot to choose the variables to aggregate.
Thanks a lot.

I would loop this.
If you use a Column List Loop Start and end this with Loop End (Column Append) you have the same effect and do not need to hack the flow variables into the pivoting node.

Thank you. I’ll try it later :slight_smile:

1 Like

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.