cURL / Mailing Lists / curl-library / Single Mail

curl-library

RE: Lingering Connections?

From: Alexis Oldroyd <alexis.oldroyd_at_gmail.com>
Date: Thu, 2 Oct 2008 17:58:22 -0500

I'll explain my app a bit more. I am using the multi interface inside an
HTTPManager class. The class is used by many threads, and they call
"BeginGetRequest" and BeginPostRequest" member on the class to start an
action. A callback is then called when the request is finished. All the
state for a specific HTTP request is stored inside an object of type
HTTPTransaction in a list that I use to keep track of active requests.

In my BeginXXXRequest functions I call CURL * curl = curl_easy_init(); and
that handle gets stored in the HTTPTransaction. On the handle I set the
options as followed ( this is for BeginGetRequest )
        curl_easy_setopt ( curl, CURLOPT_URL, tran->FullPath.c_str() );
        curl_easy_setopt ( curl, CURLOPT_HEADER, 0 );
        curl_easy_setopt ( curl, CURLOPT_HTTPGET, 1 );
        curl_easy_setopt ( curl, CURLOPT_HEADERFUNCTION,
HTTPManager::ReceiveHeaderData );
        curl_easy_setopt ( curl, CURLOPT_HEADERDATA, tran );
        curl_easy_setopt ( curl, CURLOPT_FOLLOWLOCATION, 1 );
        curl_easy_setopt ( curl, CURLOPT_WRITEFUNCTION,
HTTPManager::ReceiveData );
        curl_easy_setopt ( curl, CURLOPT_WRITEDATA, tran );
        curl_easy_setopt ( curl, CURLOPT_NOPROGRESS, false );
        curl_easy_setopt ( curl, CURLOPT_PROGRESSFUNCTION,
HTTPManager::ProgressUpdate );
        curl_easy_setopt ( curl, CURLOPT_PROGRESSDATA, tran );
        curl_easy_setopt ( curl, CURLOPT_MAXCONNECTS, 0 );
*NEW*
        curl_easy_setopt ( curl, CURLOPT_FRESH_CONNECT, 1 );
*NEW*
        curl_easy_setopt ( curl, CURLOPT_FORBID_REUSE, 1 );
*NEW*

I also have a function on a timer that calls curl_multi_perform ( I added
the easy handle to the multi interface ) from my window message loop idle
time. If this returns true, I request more idle time.

        CURLMcode performReturn = curl_multi_perform ( mCurlMHandle,
&curlRunningCount );

        // curl has been busy and needs to be poked again immediatly after
this call.
        if ( performReturn == CURLM_CALL_MULTI_PERFORM )
                moreProcessing = true;

        // Theres a few events running right now, we should probably query
them again.
        if ( curlRunningCount > 0 )
                moreProcessing = true;

        CURLMsg * msg;
        while ( true )
        {
                // process out multi handle and see whats going on?
                msg = curl_multi_info_read ( mCurlMHandle, &curlRunningCount
);

                // nothing going on break out of here.
                if ( msg == NULL )
                        break;
                // curl finished on a request...
                if ( msg->msg == CURLMSG_DONE )
                {
                        *** SNIP ***
                        No curl calls, does callbacks etc to inform the
caller what has gone on.

                }
        }

        return moreProcessing;

The issue I'm experiencing is that without the CURL_OPTIONS marked as
**NEW**, curl will maintain a socket to the server open until the server
tells it to terminate. I've been doing a bit of digging and at the end of
where I snipped the code for watching if curl signals CURLMSG_DONE I delete
an object of type HTTPTransaction which contains the state for an entire
BeginXRequest and when that object is deleted, its destructor calls
curl_easy_cleanup on the curl easy handle that was provided for that
transaction.

Can I reuse that curl_easy_handle? My thought is that I will in the
HTTPManager maintain a pool of curl_easy_handles with a time they are last
used. I'll have a timer, maybe right before perform is executed, that if
they haven't been used in Y amount of time I'll call curl_easy_cleanup on
them then.

If I went with this easy handle reuse scheme, I believe I will have to keep
the curl flags I marked with **NEW**. If I didn't, I believe I would end up
with the situation I am now where even though cleanup is called on every
curl easy handle that is used for every http transaction, curl reuses it
internally and doesn't close it on the call to cleanup. If I set the options
I have now found it seems that it kills the socket immediately when cleanup
is called which would be great for me if it still allowed reuse over
multiple requests even though I disabled curls internal reuse of the
handles.

-----Original Message-----
From: curl-library-bounces_at_cool.haxx.se
[mailto:curl-library-bounces_at_cool.haxx.se] On Behalf Of Daniel Stenberg
Sent: Thursday, October 02, 2008 4:14 PM
To: libcurl development
Subject: Re: Lingering Connections?

On Thu, 2 Oct 2008, Alexis Oldroyd wrote:

> Apparently the HTTP sessions are never disconnecting, and the web server
is
> having to time them out after 120 seconds. I believe these connections are

> completely idle. How do I prevent curl from leaving these connections open

> when they are not in use.

First, it has to leave them so that subsequent uses can be persistantly
using
the same ones again.

Then, you can close the particular handle that holds the connection and all
currently open connections will be closed. Or you can mark the individual
connection with FORBID_REUSE and then they will be closed immediately after
the transfer is done.

-- 
  / daniel.haxx.se
Received on 2008-10-03