On Tue, 22 Sep 2009, johansen_at_sun.com wrote:

>>
>> "A malicious server could cause libcurl to download an infinite amount of
>> data, potentially causing all of memory or disk to be filled. Setting the
>> CURLOPT_MAXFILESIZE_LARGE option is not sufficient to guard against this.
>> Instead, the app should monitor the amount of data received within the
>> write or progress callback and abort once the limit is reached. "
>
> It might be worthwhile to submit this question to the curl-library list
> too. I would be interested in the particulars about just how this might
> happen. The documentation for CURLOPT_MAXFILESIZE_LARGE seems to imply
> that libcurl only checks the content-length header if it exists. One
> would need to perform a transfer where no content-length header is
> present in order to escape the MAXFILESIZE check.

well there is the problem in the header, too.

arguably you could be handed an endless header and obliterate all memory.
I put a configurable max header size (default to 2mb) in place and work
from there.

> I used the multi interface, so my approach may not map exactly to what
> you're doing. However, I created an object that contained a bunch of
> information about the transfer. This object also contained the
> progress_callback(), so progress_callback(self, ...) always referred to
> the transfer that was in progress. I made sure that the easy handle had
> a reference to this object. When you're in your error loop, processing
> the easy handles that have failed, look at the object that you assigned
> to the handle. If the reason for the transfer's abort was saved in that
> object, then all you would have to do is check the code (transfer too
> large, aborted by user, whatever) and then raise the error that you
> actually wanted to raise when you aborted the callback.
>

That does make sense and it is what I ended up doing - but it felt a
little hacky.

I also wish there was a nice way to go from errorcode->string in the
python library.

thanks for the help.
-sv

_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-python
Received on 2009-09-23