Hi @Salah_Online. I just updated my TopicKR workflow on the KNIME Hub. It now includes components that calculate topic coherence via normalised pointwise mutual information (NPMI) and via the conditional probability of successive topic terms. The latter approach is described in a paper by Mimno et al. I can’t remember right now where the NPMI method is described, but I think I have seen it mentioned in a few places. It makes intuitive sense, and in the few tests that I have done, I think it produced better results than the conditional probability method. (Note also that I did some experiments to normalise the conditional method, but I am not sure if they are mathematically sound!)
More generally, I have not had anyone review the calculations in the wokflow, so I cannot guarantee that they are free from error.
The workflow also provides components that should assist the job of generating several topic models across different parameters. I have half a blog post written to explain the workflow in detail, but never got around to finishing it, on account of becoming employed! One day in the near future, I hope to finish that post. In the meantime, please feel free to correspond with me if you want to test the workflow, so we can iron out some of the bugs that are sure to be in there. (You can email me directly via the contact details on my blog.) I’d love to see KNIME used more in the topic modelling and social science space, and this could be a chance to make this happen!
I haven’t had time to review the Python notebook yet, and I barely know any Python. But I’ll be interested to know how its coherence metric compares to the ones that I have used.
Best of luck, and stay in touch!