Find matching network nodes

KNIME Analytics Platform
1

I need to parse a log file to establish which source nodes have connected which destination nodes in a network.Each node has a unique ID. I can create a network diagram but with nearly 100'000 nodes and more than 40 million rows in the log file, the network diagram is useless. 

 A simplified version of the log file columns is this:

 

row_id,sNodeId,dNodeId,dtgStart,dtgStop,mbSent,mbRec,mbTtl

 

What I need to do is find which sNodeId's (Source Node) has connected to which dNodeID's (Destination Node).

 

Using a Value Counter node it is easy enough to see how many sNodeId's exist in the network, what the node id's are, and how many connections each has made, but what I have not been able to establish is which dNodeId's each node has connected to. 

 

Can anyone help?

 

tC/.

 

2

If I understood correctly, this should be as simple as using a Group By node, where you group by source+destination and add a counter.

3

Thanks... your suggestion worked like a charm.

But as with all things, there is an evolution and now I need to calculate the sum the mbTtl column to add a weight to the link between edge node. Do you have any ideas how I could do this?

 

tC/.

4

Beside counting, the Group By node supports summing as well as aggregation method. You can define arbitrary many aggregation operations per column.

Best,
Philipp