Hi all,
i stumbled a across a bigger problem for myself yesterday: I don’t get it done to do a calculation like RowCount for a table and store it in FlowVariable so that i can use it for other calculations.
What i tried is i take a Column Expressions Node which just calculates RowCount of a table “old”
Then i thought this is easy so i take “Table Row to Variable” to make a Flow Variable “RowCountOld” out of it so that i then can use this number for a Rule-based Row filter.
but this didn’t work the flow Variable is not shown in the lilst of Flow Variables, trying to just type in the Name in the formula didn’t work neither?
So the Table Row to Variable didn’t work. I don’t know why.
But my goal actually is more this: How can generally do a calculation in a table and store this in Flow Variable to use this later on. Because its a bit wasterful to make out of a simple number like 500 a column in a table that repeats over potentially millions of rows…
So is there a way in Knime to do a calculation with Data and just save it “flat” in a flow Variable that i then can use later on.
i experimented quite a lot yesterday but it seemed a bit like “seperate worlds” Data and FlowVariables…
Any input welcome!
I’m not sure if this is the “right” answer, but I would use GroupBy (with no “Group Column(s)” specified on the “Settings” tab) to get the count of rows as a side-flow as this will then have a data table of just a single value.
You can then put this value into a variable. My preference is actually to do this with a java snippet rather than the “Table Row to Variable” or “Table Column to Variable” as whilst it is a few more clicks, and I have to add a line of code, I feel it gives more control over variable naming.
Another thought i had with the Groupby how is it with Missing values. Do you only Count Columns where you know that there are no missing values i saw that you didn’t check Missing values. Another thing is the default for Groupby is 10.000 unique values for a group has that caused trouble so far?
Hi @TotalDataLoss , I think your error is possibly just a missing ; at the end of the line, which is required by java,
@bruno29a’s component looks useful though if he’s already got one pre-packaged. Always more than one way to crack a nut, and table dimensions is probably more efficient than groupby, and something I hadn’t yet thought of, so something more for the recipe book!
@bruno29a : Nice one with the table dimension this is for the specific task at very good. The component feature comes in handy too. Thx for this!!
But what is still welcome, i am still looking if there is a generic way how to calculate something with data which has a single number as an endresult and then store it in a Flow Variable. So basically i would be looking for something as flexibel als the column expressions node, but with the possibility to store an endresult like rowcount in a flow variable.That would be awesome.
@takbb: Oh man - i guess have tired eyes today. Now it ran perfectly. Thanks i don’t know a thing about Java as you can see. What i really like about this that with GroupBy you can make anything flat.
@TotalDataLoss glad to be of help. One of the problems with populating the flow variables from column-based nodes is of course that generally speaking, you are working with multi-row values and so as you mention, they would be useful for “single value as an endresult” but you only really get those when aggregating an entire table. That said, I agree it would be nice if you had the ability to populate variables slightly more easily than at present. I have been playing with components and took @bruno29a’s Row Count (because I like it! ) and added an icon, plus config so it can be configured in the flow with the required output variable name.
I additionally created three components for taking a single Integer, String and Double (one component for each type), and these can be added into a flow as “convenience” nodes. They don’t do anything that can’t be done already, but they are just there to keep things tidy.
The idea would be that after doing whatever node it is that returned your “single value” or “single row of values” you could then chain the “Column to Named… Variable” nodes to put them into flow vars and then join the flow variables back to the main flow.
So most of my time this afternoon I’ve been playing with “Paint” “designing” 16x16 icons!
Hi @takbb , good idea to pass what the variable name to return, given the limitations that we have on variables (no variable rename, no variable filter though as I understand it, the Knime team is looking into these features).
