Recursive Twitter search with PHP

I’ve been playing with the Twitter API recently and more specifically the Twitter search API. Sometimes the results from a Twitter search span across multiple pages and because of this, the result from a call to the Twitter search API may include a next_page variable. Using this variable we can recursively search Twitter until there are no more results. In this blogpost I’ll show you how.

I didn’t write my own Twitter search library. Instead, I extended the excellent PHPTwitterSearch library by Ryan Faerman. Don’t forget to download that and include or require it before using this class. Here is the code:

<?php
require 'PHPTwitterSearch.php';
 
class RecursiveTwitterSearch extends TwitterSearch {
    var $request_count = 0;
    var $max_request_count = 5;
    var $max_id;
 
    function recursive_results() {
        $request  = 'http://search.twitter.com/search.'.$this->type;
        $request .= '?q='.urlencode($this->query);
 
        if(isset($this->rpp)) {
            $request .= '&rpp='.$this->rpp;
        }
 
        if(isset($this->page)) {
            $request .= '&page='.$this->page;
        }
 
        if(isset($this->lang)) {
            $request .= '&lang='.$this->lang;
        }
 
        if(isset($this->since)) {
            $request .= '&since_id='.$this->since;
        }
 
        if($this->show_user) {
            $request .= '&show_user=true';
        }
 
        if(isset($this->geocode)) {
            $request .= '&geocode='.$this->geocode;
        }
 
        if(isset($this->max_id)) {
            $request .= '&max_id='.$this->max_id;
        }
 
        $response = $this->objectify($this->process($request));
        $this->request_count++;
        if ($response) {
            $results = $response->results;
            if(!empty($response->next_page)) {
                preg_match('|\?page=([0-9]*)&max_id=([0-9]*)&|', $response->next_page, $matches);
                $this->page = $matches[1];
                $this->max_id = $matches[2];
                if ($this->request_count < $this->max_request_count) {
                    $results = array_merge($results, $this->recursive_results());
                }
            }
            return $results;
        }
        else return false;
    }
 
    function max_request_count($n) {
        $this->max_request_count = $n;
        return $this;
    }
}
?>

Basically what I did is duplicate the results() function from the parent class and add some code to it. Now, recursive_results() calls the API and when the variable next_page is present it will call itself again and merge the result sets. This process keeps on going either until we’re at the last page of search results or until a certain number of requests have been made. You can set this number yourself with the function max_request_count().

Here is some example code:

<?php
$rts = new RecursiveTwitterSearch('starbucks');
print_r($rts->recursive_results());
echo 'It took us ' . $rts->request_count . ' request(s) to get this result.';
?>

Got questions? Found a bug? Ideas for optimizations? General shout-outs? I’d love to hear them! :-)

12 comments on Recursive Twitter search with PHP

  1. Great script / extension! Loving it already!

    Can I also set the max number of results per page? ($this->rpp)? This significantly influences the number of results (tweets) that are retrieved..

  2. I believe you can, Timo. Thanks for your comment :)

  3. Hi, like all your plugin, can i get help please :

    http://goo.gl/UfL0T

    thank’s

  4. Thanks,Very nice and helpful.

  5. Hi where is the post?

  6. Hello Sir,

    There was a code some day ago on this site.
    I was referring that,and now it is not here…

    Please let me know that code..

    Thanks,
    Chirag

  7. Some plugin was messing up my site. It’s working reasonably well again now.

  8. Hello Sir,

    Thank you very much…

    Regards,
    Chirag

  9. So good and Simple solution, thanks a lot!
    Miss Zu Developer – From Brazil

  10. Excellent solution. Very Helpful.
    Cheers

  11. Great post! But i wonder, how dit you embed the code in the html? On my site i’m having trouble with showing the codes

  12. Hey Tom,

    I’m using the WP-Syntax plugin. There are plenty of plugins that help you insert code into WordPress posts, just search around in the plugin directory :).

Leave a response: