Asynchronous/parallel HTTP requests using PHP curl_multi

php asynchronous curl parallel-processing curl-multi
burger picture burger · Feb 12, 2010 · Viewed 7.5k times · Source

I recently looked into the possibility of making multiple requests with curl. I may not be understanding it fully, so I am just hoping to clarify some concepts.

It's definitely a good option if you are fetching content from multiple sources. That way, you can start processing the results from faster servers while still waiting for slower ones. Does it still make sense to use it if you are requesting multiple pages from the same server? Would the server still serve multiple pages at the time to the same client?

Answer

Brent Baisley picture Brent Baisley · Feb 12, 2010

You can't do multi-threading in PHP, so you won't be able to start processing one page while the others are still being retrieve. Multi-curl won't return control until all pages are retrieved or timeout. So it will take as long the it takes for the slowest page to be retrieved. You are going from serial (curl) to parallel (multi_curl), which will still give you a big boost.

Servers will serve multiple pages to the same client up to a certain configure limit. Requesting 5-10 pages from a server would be fine.

Related questions

How do I make an asynchronous GET request in PHP?

I wish to make a simple GET request to another script on a different server. How do I do this? In one case, I just need to request an external script without the need for any output. make_request('http://…

Read More
Asynchronous Function Call in PHP

I am working on an a PHP web application and i need to perform some network operations in the request like fetching someone from remote server based on user's request. Is it possible to simulate asynchronous behavior in PHP given …

Read More
How to make asynchronous HTTP requests in PHP

Is there a way in PHP to make asynchronous HTTP calls? I don't care about the response, I just want to do something like file_get_contents(), but not wait for the request to finish before executing the rest of …

Read More