异步发送多个 goutte 请求
Sending multiple goutte requests asynchronously
这是我正在使用的代码
require_once 'goutte.phar';
use Goutte\Client;
$client = new Client();
for($i=0;$i<10;$i++){
$crawler = $client->request('GET', 'http://website.com');
echo '<p>'.$crawler->filterXpath('//meta[@property="og:description"]')->attr('content').'</p>';
echo '<p>'.$crawler->filter('title')->text().'</p>';
}
这有效但需要很长时间来处理?有什么办法可以更快吗
对于初学者来说,您的代码示例没有任何异步之处。这意味着您的应用程序将依次执行获取请求、等待响应、解析响应然后循环返回。
虽然 Goutte 在内部使用 Guzzle,但它没有使用 Guzzles 的异步功能。
要真正使您的代码异步,您需要参考 Guzzle 文档:
您上面的代码示例会产生如下结果:
require 'vendor/autoload.php' //assuming composer package management.
$client = new GuzzleHttp\Client();
$requests = [
$client->createRequest('GET', $url1),
$client->createRequest('GET', $url2),
$client->createRequest('GET', $url3),
$client->createRequest('GET', $url4),
$client->createRequest('GET', $url5),
$client->createRequest('GET', $url6),
$client->createRequest('GET', $url7),
$client->createRequest('GET', $url8),
$client->createRequest('GET', $url9),
$client->createRequest('GET', $url10),
];
$options = [
'complete' => [
[
'fn' => function (CompleteEvent $event) {
$crawler = new Symfony\Component\DomCrawler\Crawler(null, $event->getRequest()->getUrl());
$crawler->addContent($event->getResponse->getBody(), $event->getResponse()->getHeader('Content-Type'));
echo '<p>'.$crawler->filterXpath('//meta[@property="og:description"]')->attr('content').'</p>';
echo '<p>'.$crawler->filter('title')->text().'</p>';
},
'priority' => 0, // Optional
'once' => false // Optional
]
]
];
$pool = new GuzzleHttp\Pool($client, $requests, $options);
$pool->wait();
这是我正在使用的代码
require_once 'goutte.phar';
use Goutte\Client;
$client = new Client();
for($i=0;$i<10;$i++){
$crawler = $client->request('GET', 'http://website.com');
echo '<p>'.$crawler->filterXpath('//meta[@property="og:description"]')->attr('content').'</p>';
echo '<p>'.$crawler->filter('title')->text().'</p>';
}
这有效但需要很长时间来处理?有什么办法可以更快吗
对于初学者来说,您的代码示例没有任何异步之处。这意味着您的应用程序将依次执行获取请求、等待响应、解析响应然后循环返回。
虽然 Goutte 在内部使用 Guzzle,但它没有使用 Guzzles 的异步功能。
要真正使您的代码异步,您需要参考 Guzzle 文档:
您上面的代码示例会产生如下结果:
require 'vendor/autoload.php' //assuming composer package management.
$client = new GuzzleHttp\Client();
$requests = [
$client->createRequest('GET', $url1),
$client->createRequest('GET', $url2),
$client->createRequest('GET', $url3),
$client->createRequest('GET', $url4),
$client->createRequest('GET', $url5),
$client->createRequest('GET', $url6),
$client->createRequest('GET', $url7),
$client->createRequest('GET', $url8),
$client->createRequest('GET', $url9),
$client->createRequest('GET', $url10),
];
$options = [
'complete' => [
[
'fn' => function (CompleteEvent $event) {
$crawler = new Symfony\Component\DomCrawler\Crawler(null, $event->getRequest()->getUrl());
$crawler->addContent($event->getResponse->getBody(), $event->getResponse()->getHeader('Content-Type'));
echo '<p>'.$crawler->filterXpath('//meta[@property="og:description"]')->attr('content').'</p>';
echo '<p>'.$crawler->filter('title')->text().'</p>';
},
'priority' => 0, // Optional
'once' => false // Optional
]
]
];
$pool = new GuzzleHttp\Pool($client, $requests, $options);
$pool->wait();