PHP使用CURL_MULTI實現多線程采集的例子

字號:


    這兩天有一客戶定制了一個免登錄發(fā)布模塊,因為在模塊中需要涉及到很多圖片下載的問題,考慮到性能問題,所以特別寫了一個CURL_MULTI遠程采集網頁的函數,以方便以后使用,估計以后都不會使用原來的單線程curl函數去foreach了,其性能對比很明顯的。同樣獲取我的博客的十個不同網頁,curl_multi:4.5246081352234,file_get_contents:33.001797914505,將近8倍的效率,可想而知,如果在附件更多的情況下,性能差異就越明顯了,希望對您有所幫助!
    代碼如下:
    <?php
    $text = remote(array('http://www.jb51.net/','http://www.baidu.com/'));
    print_r($text);
    function remote($urls) {
        if (!is_array($urls) or count($urls) == 0) {
            return false;
        }
        $curl = $text = array();
        $handle = curl_multi_init();
        foreach($urls as $k => $v) {
            $nurl[$k]= preg_replace('~([^:\/\.]+)~ei', "rawurlencode('\\1')", $v);
            $curl[$k] = curl_init($nurl[$k]);
            curl_setopt($curl[$k], CURLOPT_RETURNTRANSFER, 1);
            curl_setopt($curl[$k], CURLOPT_HEADER, 0);
            curl_multi_add_handle ($handle, $curl[$k]);
        }
        $active = null;
        do {
            $mrc = curl_multi_exec($handle, $active);
        } while ($mrc == CURLM_CALL_MULTI_PERFORM);
        while ($active && $mrc == CURLM_OK) {
            if (curl_multi_select($handle) != -1) {
                do {
                    $mrc = curl_multi_exec($handle, $active);
                } while ($mrc == CURLM_CALL_MULTI_PERFORM);
            }
        }
        foreach ($curl as $k => $v) {
            if (curl_error($curl[$k]) == "") {
            $text[$k] = (string) curl_multi_getcontent($curl[$k]);
            }
            curl_multi_remove_handle($handle, $curl[$k]);
            curl_close($curl[$k]);
        }
        curl_multi_close($handle);
        return $text;
    }