Ultimately I am aiming to identify social media topics that will be trending in a few hours (for instance to help bloggers in writing about topics their fan base will be interested in). Not only does the streaming API allow access to a greater sample of the data than the REST API, it also supplies current tweets as opposed to the past 100 tweets of a topic which could go back as far as 2 days. I will also be using the REST API in my analysis, but getting that data was the easy part As mentioned above, I have already succeeded in getting the streaming API data using Proc HTTP. The problem is that I cannot create an automated process where the data is downloaded, imported and predictions are made if I can never get to the importing stage. Since my previous post, I have also tried using HTTPBuilder as a replacement for Proc HTTP hoping that Groovy will allow me to stop downloading at some stage and reconnect to the API later, but I keep getting a “401 Unauthorized” error. Once I figure that out I might find a solution using Groovy, but I would really appreciate other suggestions.
... View more