cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: How to do simultaneous transfer continuously in all the existing connections

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Wed, 3 Dec 2014 15:06:51 +0100 (CET)

On Tue, 2 Dec 2014, Mohanraj V wrote:

> 1. For eg., I have made 1000 connections by adding individual 1000 easy
> handles to a multi handle. After that, I did curl_multi_perform using that
> multi_handle. LibCurl had established 1000 tcp connections and transmitted
> one requests per connection .But How can i transfer requests (http get)
> continuously in all the 1000 connections concurrently.

You add a new transfer for the same host name and port number as soon as one
transfer ends.

> Is it possible using multi_perform. But multi_perform performs single
> transfer in all the handles ? Or else Any other way to achieve this ?

multi_perform performs all transfers you've added. Just add the transfers you
want.

> i) What is the maximum(approx.) possible connections ?

1, curl_multi_socket_action scales much better when going beyond hundreds of
connections.

2. You need to change so that you can do more than 1024 open sockets.

3. There is no maximum imposed by libcurl, but you will probably saturate
something when going up in the several tens of thousands.

4. the multi interface is single-threaded and to fully utilize your CPU cores
you want to use at least one thread per core

> ii) What is the maximum(approx.) requests per connection possible ?

Try it and tell us. I'm not aware of anyone posting such info anywhere and it
is also likely to be very momentary and depending on a specific machine and
network setup and conditions

> iii) Is it possible to define request rate and connection rate using
> lib-curl ?

You can limit the transfer rates, yes. You can limit the request rate yourself
by simply not adding new transfers if the previous one ended too soon.

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Received on 2014-12-03