cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Multi-threading and name resolution issues

From: Jay Sastry <jaynsastry_at_gmail.com>
Date: Wed, 3 Aug 2011 11:36:01 -0700

Thanks for your response, Dan.
I am now building my own libcurl version that is linked against c-ares for
testing.
You are right about the resolver problem, someone had changed the host name
in our configuration to its IP address, so there was no name resolution.
But, I do want to know more about the connection cache libcurl internally
may (or may not) maintain.
I want my program to be fault tolerant to certain errors, so I intercept the
CURL_COULDNT_CONNECT error and reset the handle. I would like to know why my
program cannot recover from this error after doing a curl_easy_duphandle. I
am able to connect to the server outside of my program without any problem.
Should I do curl_easy_init instead of curl_easy_duphandle? Incidentally, I
connect to two different web servers, I have not noticed any connection drop
to the local server, but connection always drops to the server that is on a
different subdomain. I am investigating all components in the middle between
the end points to track down the root cause, but I still expect my program
to recover from such errors after resetting the handle. What else can I do
in my program to make it fault-tolerant to connection drops?
Thx,

  -- Jay

On Tue, Aug 2, 2011 at 11:38 PM, Dan Fandrich <dan_at_coneharvesters.com>wrote:

> On Tue, Aug 02, 2011 at 05:48:48PM -0700, Jay Sastry wrote:
> > I am writing a C++ program that communicates with web-services; I am
> using
> > libcurl to talk HTTP with the web services. My program is multi-threaded
> and is
> > quite chatty with the web services (interacts periodically every few
> seconds).
> > For this reason, I am using a pool of CURL handles to avoid a massive
> socket
> > churn. I have read the documentation and other posts regarding caveats
> with
> > multiple threads, so I have taken care to make sure each handle is used
> by only
> > one thread at a time, although I bounce the handles between different
> threads.
> > I have not encountered any problems in this area. However, I am noticing
> > problems connecting to the web server after a few hours(I am getting a
> > CURLE_COULDNT_CONNECT error). I am suspecting this could be due to the
> name
> > resolution issue. I am using the libcurl that is baked into Centos 6 and
> so it
> > is not linked against a-res lib.
>
> If libcurl isn't using C-Ares, then it must be using the host resolver (I
> believe that the version of libcurl shipped with CentOS predates the
> threaded
> resolver). So, if there is a resolver problem, it could be with the host
> resolver. But are you sure there's a resolver problem at all? Have you
> ruled
> out a problem with the remote DNS server or a network problem in the
> middle?
> That's the most likely explanation. You could also try compiling your own
> libcurl against C-Ares for testing and see if the problem goes away.
>
> >>> Dan
> -------------------------------------------------------------------
> List admin: http://cool.haxx.se/list/listinfo/curl-library
> Etiquette: http://curl.haxx.se/mail/etiquette.html
>

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2011-08-03