Bugs item #1326306, was opened at 2005-10-14 02:31
Message generated for change (Comment added) made by bagder
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=1326306&group_id=976

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: libcurl
Group: bad behaviour
Status: Open
Resolution: None
Priority: 7
Submitted By: Amol Pattekar (pattekar)
Assigned to: Daniel Stenberg (bagder)
Summary: Socket descriptor leak with libcurl-multi

Initial Comment:

Hi,

The following test code reproduces the problem:

#include <string.h>
#include <curl/curl.h>
#include <unistd.h>

int main(void)
{
   CURL *curl_easy_handle;

   curl_easy_handle = curl_easy_init();
   if(curl_easy_handle) {
     curl_easy_setopt(curl_easy_handle,
CURLOPT_URL, "http://somehost.yahoo.com");
     curl_easy_setopt(curl_easy_handle,
CURLOPT_NOBODY, 1);
     curl_easy_setopt(curl_easy_handle,
CURLOPT_VERBOSE, 1);
     curl_easy_setopt(curl_easy_handle,
CURLOPT_DNS_CACHE_TIMEOUT, 0);

     CURLM *curl_multi_handle = curl_multi_init();

     for (int i=0; i<2000; i++)
       {
         curl_multi_add_handle(curl_multi_handle,
curl_easy_handle);

         int running_handles=1;

         while (running_handles)
           {
             CURLMcode code =
CURLM_CALL_MULTI_PERFORM;
             while (code ==
CURLM_CALL_MULTI_PERFORM)
               {
                 code = curl_multi_perform(curl_multi_handle,
                                           &running_handles);
               }

             if (running_handles)
               {
                 int maxFD;
                 fd_set readfds, writefds, exceptfds;
                 FD_ZERO(&readfds);
                 FD_ZERO(&writefds);
                 FD_ZERO(&exceptfds);
                 curl_multi_fdset(curl_multi_handle,
&readfds, &writefds, &exceptfds, &maxFD);

                 select(maxFD+1, &readfds, &writefds,
&exceptfds, NULL);
               }
           }

         curl_multi_remove_handle(curl_multi_handle,
curl_easy_handle);
       }
   }
   return 0;
}

Also,
1) "somehost.yahoo.com" resolves to two IP address.
One of the IP addresses experiences issues due to
which libcurl sees connection failures to that IP. The
other IP works well.
2) "somehost.yahoo.com" doesn't like persistent
connections, and returns "Connection: close" header
with every response.

I find that when libcurl experiences connection failure
with the first IP, it automatically tries the second one.
However, it doesn't close the socket descriptor that it
created to connect to the first IP, and this results in a
descriptor leak.

Thanks,

-Amol.

----------------------------------------------------------------------

>Comment By: Daniel Stenberg (bagder)
Date: 2005-10-22 23:42

Message:
Logged In: YES
user_id=1110

Ok, I modded my test case:

I now connect to "bad2.haxx.se:8080" and made sure I have a
local web server on 8080.

I ran the test case with valgrind, I could see outputs like

* About to connect() to bad2.haxx.se port 8080
* Trying 193.15.23.131... * STATE: CONNECT => WAITCONNECT
handle 0x1bc5bd7c:
* Connection refused
* Trying 127.0.0.1... * connected

(meaning it attempted one, failed and then tried the next
and succeeded)

valgrind does not report any leaks!

(testing curl CVS, ipv6 disabled on Linux 2.6)

How do you detect the leak in the first place? What does
valgrind tell you if you run this with a debug version?

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2005-10-20 23:21

Message:
Logged In: NO

I tried it the way you did it, and that didn't repro the problem.

It appears that in your case, the 10.* IPs point to non-
existent machines, as a result of which the connect hangs
unless a short timeout is used.

This problem can be reproduced if the 10.* IPs represent
existing machines which do not have a server listening on
port 80. In that case, the connect will come back
immediately with a failure (ECONNREFUSED, Connection
refused), and this is when the descriptor leaks.

Thanks,
-Amol.

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2005-10-20 21:47

Message:
Logged In: YES
user_id=1110

I ran it against this host name: bad10.haxx.se

It resolves to 5 IP addresses, one being 127.0.0.1 and the
rest being 10.* ones that my libcurl fails to connect to.

(To make it bearable, I reduced the default connect timeout
to 3 seconds for test purposes.)

What kind of "connection failures" do you get from the IP in
condition (1) ?

I modified your example to 1 - close both the easy and multi
handles and 2 - timeout on the select) - to make it easier
to use for memory leak tracking.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2005-10-20 21:42

Message:
Logged In: NO

Did you run the test code against a hostname that has the
following two properties:

>
>1) "somehost.yahoo.com" resolves to two IP address.
>One of the IP addresses experiences issues due to
>which libcurl sees connection failures to that IP. The
>other IP works well.
>2) "somehost.yahoo.com" doesn't like persistent
>connections, and returns "Connection: close" header
>with every response.
>

I will also look into this some more to send you more detail.

-Amol.

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2005-10-20 21:01

Message:
Logged In: YES
user_id=1110

I fail to repeat this problem.

Can you figure out any further details? Like what socket or
similar.

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2005-10-14 15:19

Message:
Logged In: YES
user_id=1110

I'll look into it asap, but it might take a week or so. I
believe this issue still remains in 7.15.0 as I can't recall
any fix that would've changed this.

----------------------------------------------------------------------

Comment By: Amol Pattekar (pattekar)
Date: 2005-10-14 02:37

Message:
Logged In: YES
user_id=1326426

Sorry, forgot to mention: I am using libcurl-7.14.1

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=1326306&group_id=976
Received on 2005-10-22