Recursive Twitter search with PHP

I’ve been playing with the Twitter API recently and more specifically the Twitter search API. Sometimes the results from a Twitter search span across multiple pages and because of this, the result from a call to the Twitter search API may include a next_page variable. Using this variable we can recursively search Twitter until there are no more results. In this blogpost I’ll show you how.

I didn’t write my own Twitter search library. Instead, I extended the excellent PHPTwitterSearch library by Ryan Faerman. Don’t forget to download that and include or require it before using this class. Here is the code:

<?php
require 'PHPTwitterSearch.php';
 
class RecursiveTwitterSearch extends TwitterSearch {
    var $request_count = 0;
    var $max_request_count = 5;
    var $max_id;
 
    function recursive_results() {
        $request  = 'http://search.twitter.com/search.'.$this->type;
        $request .= '?q='.urlencode($this->query);
 
        if(isset($this->rpp)) {
            $request .= '&rpp='.$this->rpp;
        }
 
        if(isset($this->page)) {
            $request .= '&page='.$this->page;
        }
 
        if(isset($this->lang)) {
            $request .= '&lang='.$this->lang;
        }
 
        if(isset($this->since)) {
            $request .= '&since_id='.$this->since;
        }
 
        if($this->show_user) {
            $request .= '&show_user=true';
        }
 
        if(isset($this->geocode)) {
            $request .= '&geocode='.$this->geocode;
        }
 
        if(isset($this->max_id)) {
            $request .= '&max_id='.$this->max_id;
        }
 
        $response = $this->objectify($this->process($request));
        $this->request_count++;
        if ($response) {
            $results = $response->results;
            if(!empty($response->next_page)) {
                preg_match('|\?page=([0-9]*)&max_id=([0-9]*)&|', $response->next_page, $matches);
                $this->page = $matches[1];
                $this->max_id = $matches[2];
                if ($this->request_count < $this->max_request_count) {
                    $results = array_merge($results, $this->recursive_results());
                }
            }
            return $results;
        }
        else return false;
    }
 
    function max_request_count($n) {
        $this->max_request_count = $n;
        return $this;
    }
}
?>

Basically what I did is duplicate the results() function from the parent class and add some code to it. Now, recursive_results() calls the API and when the variable next_page is present it will call itself again and merge the result sets. This process keeps on going either until we’re at the last page of search results or until a certain number of requests have been made. You can set this number yourself with the function max_request_count().

Here is some example code:

<?php
$rts = new RecursiveTwitterSearch('starbucks');
print_r($rts->recursive_results());
echo 'It took us ' . $rts->request_count . ' request(s) to get this result.';
?>

Got questions? Found a bug? Ideas for optimizations? General shout-outs? I’d love to hear them! :-)

10 comments on Recursive Twitter search with PHP

  1. Timo zegt:

    Great script / extension! Loving it already!

    Can I also set the max number of results per page? ($this->rpp)? This significantly influences the number of results (tweets) that are retrieved..

    June 9th, 2011

  2. admin zegt:

    I believe you can, Timo. Thanks for your comment :)

    June 9th, 2011

  3. wall zegt:

    Hi, like all your plugin, can i get help please :

    http://goo.gl/UfL0T

    thank’s

    December 25th, 2011

  4. Purohit zegt:

    Thanks,Very nice and helpful.

    February 22nd, 2012

  5. Chirag zegt:

    Hi where is the post?

    February 29th, 2012

  6. Chirag zegt:

    Hello Sir,

    There was a code some day ago on this site.
    I was referring that,and now it is not here…

    Please let me know that code..

    Thanks,
    Chirag

    February 29th, 2012

  7. admin zegt:

    Some plugin was messing up my site. It’s working reasonably well again now.

    February 29th, 2012

  8. Chirag zegt:

    Hello Sir,

    Thank you very much…

    Regards,
    Chirag

    March 1st, 2012

  9. Zu zegt:

    So good and Simple solution, thanks a lot!
    Miss Zu Developer – From Brazil

    June 29th, 2012

  10. Net66 zegt:

    Excellent solution. Very Helpful.
    Cheers

    March 4th, 2013

Leave a response: