Folks - a newbie to text processing and as such have a naive question. I am using the n-gram creator to pick out ngrams and their frequencies. I am not splitting the sentences/documents into bag of words which I interpret as every record being a unique document. I assumed that sentence frequency is the # of sentences with the n-gram, document frequency is the # of documents with sentences that have the n-gram and corpus frequency the total number of documents with sentences containing the ngram Clearly my assumption on the definitions does not seem to be accurate If that were true then I am trying to wrap my head around why the sentence frequency i.e. total number of sentences would be < document frequency and likewise why document frequency which is total number of documents be lower than the corpus frequency Attached is a sample output. Would be very grateful if someone could point me in the right direction [ngram%20output]

nGram Creator

MH July 10, 2018, 6:34am 2

There you go. Explained nicely in a previous Blog post on the same discovery:

I had the same issue once before. Maybe better explanations in the Node description could help.