On Jul 17, 2009, at 12:04 PM, Heston Liebowitz wrote:
> I'm running into a strange issue using libcurl with ruby bindings.
> I've tested with both patron(i) and curb(ii) to the same effect,
> leading me to believe this is an issue with libcurl and not the ruby
> I have an array of URLs which I want to loop over and GET in turn.
> About halfway through the array, one of the URLs results in a
> malformed packet being sent back from the server (according to
> Wireshark). After that point, all subsequent GETs have their DNS
> lookups perturbed in some way. For example, here is a subset of the
> URLs with the proper DNS resolutions:
> 184.108.40.206 << http://povertyenvironment.net/misc/content/welcome/opera-music.html
> 220.127.116.11 << http://bioportfolio.com/bookshop/ASIN/B000IOEVAS
> 18.104.22.168 << http://vitaminshoppe.com/search/en/query.jsp?
> The first URL in the list returns a malformed response ("TCP
> Previous Segment Lost" according to Wireshark). After this, GETing
> the following two URLs raises "HostResolutionError" exceptions. If I
> omit the first URL though, the next two resolve and fetch correctly.
> I'm using curl 7.19.3 on Mac OS 10.5.7 and ruby 1.8.6. I haven't
> tested with the latest releases of cURL.
> Hopefully this is enough info to reproduce the problem. Thanks for
> any insight.
> i. http://fiatdev.com/2009/07/03/announcing-patron-a-ruby-http-client-library
> ii. http://curl.haxx.se/libcurl/ruby/
Update: as an experiment I tried setting CURLOPT_DNS_CACHE_TIMEOUT to
0 to disable the DNS cache. Lo and behold the problem disappeared. It
seems to me that the response for the URL "http://povertyenvironment.net/misc/content/welcome/opera-music.html
" is corrupting the DNS cache in some way. That's about as far as I
can take it though, not being all that familiar with cURL internals.
Received on 2009-07-17