Recursive Twitter search with PHP

I’ve been playing with the Twitter API recently and more specifically the Twitter search API. Sometimes the results from a Twitter search span across multiple pages and because of this, the result from a call to the Twitter search API may include a next_page variable. Using this variable we can recursively search Twitter until there are no more results. In this blogpost I’ll show you how.

I didn’t write my own Twitter search library. Instead, I extended the excellent PHPTwitterSearch library by Ryan Faerman. Don’t forget to download that and include or require it before using this class. Here is the code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
<?php
require 'PHPTwitterSearch.php';
 
class RecursiveTwitterSearch extends TwitterSearch {
    var $request_count = 0;
    var $max_request_count = 5;
    var $max_id;
 
    function recursive_results() {
        $request  = 'http://search.twitter.com/search.'.$this->type;
        $request .= '?q='.urlencode($this->query);
 
        if(isset($this->rpp)) {
            $request .= '&rpp='.$this->rpp;
        }
 
        if(isset($this->page)) {
            $request .= '&page='.$this->page;
        }
 
        if(isset($this->lang)) {
            $request .= '&lang='.$this->lang;
        }
 
        if(isset($this->since)) {
            $request .= '&since_id='.$this->since;
        }
 
        if($this->show_user) {
            $request .= '&show_user=true';
        }
 
        if(isset($this->geocode)) {
            $request .= '&geocode='.$this->geocode;
        }
 
        if(isset($this->max_id)) {
            $request .= '&max_id='.$this->max_id;
        }
 
        $response = $this->objectify($this->process($request));
        $this->request_count++;
        if ($response) {
            $results = $response->results;
            if(!empty($response->next_page)) {
                preg_match('|?page=([0-9]*)&max_id=([0-9]*)&|', $response->next_page, $matches);
                $this->page = $matches[1];
                $this->max_id = $matches[2];
                if ($this->request_count < $this->max_request_count) {
                    $results = array_merge($results, $this->recursive_results());
                }
            }
            return $results;
        }
        else return false;
    }
 
    function max_request_count($n) {
        $this->max_request_count = $n;
        return $this;
    }
}
?>

Basically what I did is duplicate the results() function from the parent class and add some code to it. Now, recursive_results() calls the API and when the variable next_page is present it will call itself again and merge the result sets. This process keeps on going either until we’re at the last page of search results or until a certain number of requests have been made. You can set this number yourself with the function max_request_count().

Here is some example code:

1
2
3
4
5
<?php
$rts = new RecursiveTwitterSearch('starbucks');
print_r($rts->recursive_results());
echo 'It took us ' . $rts->request_count . ' request(s) to get this result.';
?>

Got questions? Found a bug? Ideas for optimizations? General shout-outs? I’d love to hear them! :-)

12 thoughts on “Recursive Twitter search with PHP

  1. Great script / extension! Loving it already!

    Can I also set the max number of results per page? ($this->rpp)? This significantly influences the number of results (tweets) that are retrieved..

  2. Hello Sir,

    There was a code some day ago on this site.
    I was referring that,and now it is not here…

    Please let me know that code..

    Thanks,
    Chirag

    1. Hey Tom,

      I’m using the WP-Syntax plugin. There are plenty of plugins that help you insert code into WordPress posts, just search around in the plugin directory :).

Leave a Reply

Your email address will not be published. Required fields are marked *