'Raw' Twitter Streaming feed

KNIME Analytics Platform
1

Is it possible to conifgure the Twitter Streaming node to return the 'raw' twitter feed?  i.e. I'd like to return the 1% (ish) sample & write it to a database for retrospective analysis, but the current Twitter Streaming node requires a keyword. 

From my reading I'm expecting around 3,000 tweets a minute/180,000 tweets an hour but I'm getting a lot less than that due to keyword inclusion.

So far I've tried to be creative with the keyword, for example

Keyword: "@" AND NOT "@"

and I was only getting 10K in an hour.

Am I missing something? Is there a better way?

Is anyone doing a long-term pull of the Twitter Stream using KNIME?

Any help much appreciated.

Kind Regards,

Andrew

 

2

Hi Andrew,

The way I understand the Twitter Streaming API, I'm not sure if it is actually possible to use it without a search query at all (even outside KNIME). Have you been able to do this in other applications?

Cheers,

Roland