Column Appender - Useless?

Hello everybody,

I want to join two tables of equal length which share no column names and which have different row IDs. Isn't that exactly what the Column Appender node is supposed to do?

"Execute failed: Tables contain non-matching rows or are sorted differently, keys in row 0 do not match"

Best,
Marc

P.S.:

I know how to workaround that, so I don't seek a solution for a specific problem. I just want to discuss the node mentioned above.

If you have the R Scripting nodes (Community Nodes) installed (and an R server running), you could do this with an R Snippet (2:1) node.

Connect your 2 data tables and paste the following into the Script Editor tab of the snippet:

if (nrow(kIn1) != nrow(kIn2)) stop('tables not same length')
rOut <- cbind(kIn1,kIn2)

It assumes there are no columns common to both tables.

The column appender needs RowIDs to match in order to execute.  If you use Row ID > Column appender it should work as expected. 

Aaron

I agree with Marc.

joiner node will take rowIDs for a match in just as easy process. So I don't see the point of this node.

i would assume that column appender is to be so quick and easy that it can join equal length tables with different rowids. Otherwise it's no easier than using joiner node.

simon.

 

Let my agreement join the chorus.

The node should just be an append, like the row append, completely ignoring the rowids.

This node should have an option to just append the columns regardless of the rowids.

-- Scott

The Column Appender node should simply use the ROWINDEX to join two tables instead of the ROWID.

Best,
Marc

Or the joiner could have that option, ridding us of an already unpopular node. :)

Cheers
E

I do like the concept of this column appender node, it's quick and easy, especially for novice users, not having to think about inner/outer/full join etc. it just needs implementing a little better.

the comment by Marc about using ROWINDEX instead of RowID looks a good choice. With an option in the node about which table RowID to use in the table output from the initial two tables (if they are not the same). This option will give it an advantage over the joiner node, as you can then retain one of the original table RowIds.

simon.

The difference between joining on RowIDs and using the RowID > Column Append becomes apparent when working with lots of data.  To illustrate this, try joining 2x1million row tables by RowID and with the Column Appender.  The join takes a full minute on my system, while the column append takes ~5 seconds.  

As for the matching RowID requirement, I agree in principle that it would be nice to drop, but I would need to look into how this would effect performance and the complexity of the code, which are two areas where the current implementation is reasonably strong. 

Thanks for the feedback though!

The think the joiner is missing and what could make the append node very useful, is that the joiner can be used in loop because the structure of the input node changes. The append note doesn't have problems with that.

I'm try to get several properties from a column and have a hard time doing that in a loop.

So I have (I'm not the designer of this !):

id property value
1 logp -4
1 logS -3
2 logp -3
  many more properties  

So now I want to create:

id logP logS
1 -4 -3
2 -3 NA

So I created a loop, appender node won't work because different row sizes. Joiner won't work because of changing data structure. "loop end append column" will create all rows again.

Any suggestions?

 

 

Hi Peter,

this is a case for the pivoting node.

In your case you select the id as group, the property as pivots and e.g. the mean of the value in the options.

Cheers, Iris

I was able to solve this for outputting lists with the loop end column append node by adding a RowID node and setting it to replace the RowID. That way the list for each iteration aligns with the next and the lists keep their sort order. Not a universal solution, but it may help someone.