PHP 加密页面的卷曲

Question

我对这个网站做了 php curl http://www.hoovers.com/company-information/company-search.html

但是它返回了 404。看起来像是加密的东西什么的。

你能提供一些关于这个问题的线索吗？谢谢

        // Get cURL resource
    $curl = curl_init();
    // Set some options - we are passing in a useragent too here
    curl_setopt_array($curl, array(
        CURLOPT_RETURNTRANSFER => 1,
        CURLOPT_URL => 'http://www.hoovers.com/company-information/company-search.html',
    ));
    // Send the request & save response to $resp
    $resp = curl_exec($curl);
    // Close request to clear up some resources
    curl_close($curl);

Answer 1

看起来他们的网络服务器正在拒绝基于 HTTP headers 的请求。或者它也可能在应用程序级别。试试这个

<?php
// Get cURL resource
$curl = curl_init();
// Set some options - we are passing in a useragent too here
curl_setopt_array($curl, array(
    CURLOPT_RETURNTRANSFER => 1,
    CURLOPT_HEADER=>1,
    CURLOPT_URL => 'http://www.hoovers.com/company-information/company-search.html',
    CURLOPT_HTTPHEADER=> array(
        'User-Agent: Mozilla/5.0 (Windows NT 6.3; rv:36.0) Gecko/20100101 Firefox/36.0', 'Accept-Language: en-US,en;q=0.5'
        )
    ));

// Send the request & save response to $resp
$resp = curl_exec($curl);

// Close request to clear up some resources
curl_close($curl);

//debug
print_r($resp);
?>

PHP 加密页面的卷曲

PHP Curl for encrypted pages

php

encryption

curl

web-crawler