curl / Mailing Lists / curl-library / Single Mail

curl-library

Re: persistent connections and dns changes

From: Justin Karneges <justin_at_fanout.io>
Date: Thu, 8 Jun 2017 22:17:30 -0700

On Thu, Jun 8, 2017 at 2:20 AM, Rainer Canavan <rainer+curlusers_at_7val.com>
wrote:

> On Thu, Jun 8, 2017 at 10:34 AM, Daniel Stenberg <daniel_at_haxx.se> wrote:
> > On Wed, 7 Jun 2017, Justin Karneges wrote:
> [...]
> >> Any suggestions on how to deal with this problem? It doesn't look like
> the
> >> API has a way to destroy existing connections when the multi interface
> is
> >> used (other than tearing down the entire multi instance, which I
> suppose I
> >> could do as a last resort).
> >
> >
> > If there was an API to tear down a single connection (which has been
> > suggested before and I would agree to), how would your application know
> when
> > to call that for this situation? It seems it would need some other means
> to
> > figure that out...
>
> Could this specific problem be solved by handling DNS lookups outside
> libcurl,
> injecting the results via CURLOPT_RESOLVE and just plain not reusing the
> easy handle if the lookup results differ? As you've argued, this behaviour
> isn't
> really desirable in the general case, but we're basically doing the same
> for
> whenever certain configuration options are changed in our application.
>

FWIW, we are handling DNS resolves externally and explicitly setting the
target IP with CURLOPT_RESOLVE, although I'm not sure if that should play
into this.

Daniel makes a good point about how IPs may rotate even if nothing was
moved. Another issue I just realized is that it may be desirable to keep
connections longer than the DNS TTL, since some people use very low TTLs.

What may be best here is a persistent connection maximum time. Maybe this
could be a new option, with a large default value like 24 hours? This way
connections will eventually get unstuck in a reasonable period of time
(reasonable in server migration terms).

I'm wondering if I could implement in this behavior in our app without
needing to modify libcurl, by using a multi instance per domain, and
destroying a multi instance after it's been around for the maximum time. Is
it efficient to have lots of multi instances? Right now we have just one
master multi instance.

Justin

-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2017-06-09