Hi, i am working on a project where i want to find Trends(Topics that emerge have a plateau phase and "disappear" again), how can KNIME help me finding this trends?
The data is available as e-mails.
Best regards
jb
Hi jb,
first you have to get the text data into some sort of sensible form. The KNIME text mining extensions will help you do that. Then it is a question of how you want to define trends - you can simply do a BOW operation and frequencies, decide what types of terms you are interested in, then set up some BINS to group frequent terms. Over time, you would rerun with new data and rescore the words - "old" topics would decline and "new" topics would appear. Effectively you would be sampling over time, or establishing a rolling "window" of data.
There is a new technical paper on text mining on the KNIME.com website that walks through how to do a lot of this.
If you keep the old bin categories/scores, then you can then choose from a variety of ways to visualize this over time - or use a wordcloud to get that "at the moment" view.
That is the straightforward way. An even better way is if you have the information about who is sending/receiving, etc. That is where we bring in the power of networking analytics (also available via KNIME) to add not just word counts, but actual authority values of the folks writing and responding. More advanced but if you are interested there is a great white paper available on the KNIME.com website combining text and network analytics. Where in that paper we focused on the individuals, the concept works for terms as well.
hope this helps!
phil
Hi Phil,
thanks for the hints, that was a big help!
regards
jb