curl-and-python

Problems with multi and MAX_RECV_SPEED_LARGE

From: Chris Elston <celston_at_katalix.com>
Date: Fri, 06 Aug 2010 12:55:11 +0100

Folks,

I'm having problems using the MAX_RECV_SPEED_LARGE option with a
CurlMulti handle. My app is reasonably complex, but luckily the problem
is easily reproduceable with a simple change to the retriever-multi.py
example.

I've attached my modified retriever-multi.py: I've added a setopt to
limit the speed with MAX_RECV_SPEED_LARGE, and rather than using the
CurlMulti.select function, I'm using select.select because
CurlMulti.select hides some of the detail I'm interested in.

The list of fds to select on is obtained via CurlMulti.fdset, and the
timeout value is got via CurlMulti.timeout. My understanding of
curl_multi_timeout is that:

-1 is no timeout (i.e.: just select on the fds given, and never timeout)
0 is no select (i.e.: skip straight to the next CurlMulti.perform call)
and any other positive value is a time in milliseconds to use with the
select call.

If you run the code as I have it, it very quickly approaches a situation
with an empty fd set and no timeout, and the transfer stalls. If you
disable the setopt of MAX_RECV_SPEED_LARGE this doesn't happen, the
fdset always contains something to select on.

As an additional (although somewhat confusing) datapoint, if you comment
in the lines which apply a low speed timeout (which are also required in
my app):

    #c.setopt(pycurl.LOW_SPEED_LIMIT, 100)
    #c.setopt(pycurl.LOW_SPEED_TIME, 120)

The suggested timeout value returned by CurlMulti.timeout() is never -1,
but it does quickly go up to 120 seconds. Once this timeout has expired
the transfer happens at full-rate (presumably trying to make up the
average speed to the limit value), before repeating the same thing it
does at the beginning.

Can anyone shed some light on what I'm doing wrong here?

Thanks,

Chris.

_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-python

Received on 2010-08-06