cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Multi handle connection pool (Re: max download/upload speed setting)

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Mon, 6 Jul 2009 23:45:24 +0200 (CEST)

On Mon, 6 Jul 2009, Ragnar Lonn wrote:

>> It works fine anyway. This is because when the speed limit is reached,
>> libcurl takes away the socket of that too-fast-transfer from the list of
>> sockets the app will wait for until the speed is again below the threshold
>> and then it gets reinstated.
>
> Aha, ok. And the speed limit is per easy handle then also?

Yes. You set the option per easy handle, so it will be applied to the
connections that particular easy handle will end up using.

> Well, if the speed limit works just as it does in easy mode, then one reason
> for disabling connection pooling just disappeared. I have another reason
> though, maybe :-) We want to simulate a number of web browsers, each using a
> number of TCP connections to fetch objects over HTTP. We want to limit the
> amount of TCP connections available to *one* of our simulated browsers, but
> the CURLOPT_MAXCONNECTS option will not work in multi mode.
> CURLMOPT_MAXCONNECTS, on the other hand, limits the whole multi handle. We
> want to limit an easy handle within the multi handle, but that isn't
> possible.

Ah right.

I think it wouldn't be too hard of a job to get supported. We used to do the
connection pool on a per-easy-handle basis in the past even while in multi
mode, and since we already support it per-easy-handle when the easy interface
is used it is mostly a case of making a new "mixed mode". Although of course I
don't see many other use-cases for such a mode apart from an attempt to more
closely simulate a browser's connection set.

Thinking about that further, would it really improve the browser-emulation
much? I mean, won't you still need even further control to really make the
connections look and behave as if they were made with a recent browser?

> I suppose we could use one multi handle for each simulated web browser, and
> set CURLMOPT_MAXCONNECTS to the number of TCP sessions a single simulated
> browser client may use, but it seems a bit messy when you want to simulate
> thousands of browser clients and I'm a little worried that memory
> consumption will be high, but I haven't tried it - maybe it won't be so bad?

I don't know what "bad" is for you, and I actually don't even know the real
amount of memory multi and easy handle will use etc. I think some actual
testing is the best way to figure this out.

-- 
  / daniel.haxx.se
Received on 2009-07-06