Join tables - Ngrams

Hi!
I’m working in text processing and I want to know the TFIDF value of some bigrams.
Right now with what I have are two tables that have a structure similar to the following:

Table#1
Document Ngrams TF
Document1 w1 w2 3
Document1 w3 w4 4
Document2 w1 w2 1

Table#2
Ngrams IDF
w1 w2 2.9
w3 w4 1

I want to arrive at a structure like:

Table#3
Document Ngrams TFIDF
Document1 w1 w2 32.9
Document1 w3 w4 4
1
Document2 w1 w2 1*2.9

I was thinking of doing a script from R or Matlab where I read both tables and execute the operations I want but I do not know how to join the two tables without mixers, that is, I want a structure like

Document Ngrams(Tabla#1) TF Ngrams(Tabla#2) IDF
Document1 w1 w2 3 w1 w2 2.9
Document1 w3 w4 4 w3 w4 1
Document2 w1 w2 1

I want a table like the one exposed because the Matlab Snniped Node only allows you to add a table … so I ask the form.
If someone recommends another way to get Table # 3 in a much simpler way I would like to hear it.

Thank you very much}
Manuela

Hi Manuela,

You could use a GroupBy node after Table#1 and group on the Ngrams and count the occurrences, which would result in a DF value. Then use the Math Formula node to compute IDF from if. Then use the Joiner node to join the resulting table to Table#1. As join cols use the Ngrams cols. Only join the IDF column. In the dialog on the node you can specify which cols will be joined. After that use again the Math Formula node to multiply TF and IDF to get TFIDF.

Hope that helps.

Cheers, Kilian

Thank you so much for your advice. I have done what you recommended and it worked.

Thank you.

Manuela