curl-and-python

Re: bug in Curl.reset()?

From: <johansen_at_sun.com>
Date: Wed, 15 Apr 2009 16:15:05 -0700

I have a pair of patches for the aforementioned issues. The first seems
to solve the reset problem that I encountered. The second allows pycurl
to build against versions of OpenSolaris that don't have the static-libs
option for curl-config. I'm assuming that the second patch isn't right
for general consumption; however, the reset fixes seem to hold water.

-j

On Sat, Mar 07, 2009 at 10:07:53PM +0100, Kjetil Jacobsen wrote:
> i'll try releasing a bugfix release sometime next week.
>
> regards,
> kjetil
>
> On Fri, Mar 6, 2009 at 2:45 AM, <johansen_at_sun.com> wrote:
> > Glad I could help.  Do you have any idea when the next release might be?
> >
> > I've also run across a build problem in the process of compiling pycurl
> > on OpenSolaris.  The build script tries to invoke curl-config with the
> > --static-libs option, but our version of libcurl doesn't seem to have
> > that option available.  I'm not quite sure what the best solution is,
> > perhaps to try with static-libs, and if popen returns with an exit
> > status of 1, try without static libs.  We worked around the problem by
> > simply dropping the invocation with static-libs, but that's probably not
> > the right solution for all platforms.
> >
> > Thanks again for your help!
> >
> > -j
> >
> > On Thu, Mar 05, 2009 at 10:28:40PM +0100, Kjetil Jacobsen wrote:
> >> hi and thanks for the detailed bug report!
> >>
> >> i think your suggestion for a fix is right, i'll fix this for the next release.
> >>
> >> kjetil
> >>
> >> On Thu, Mar 5, 2009 at 3:48 AM,  <johansen_at_sun.com> wrote:
> >> > I seem to have run afoul of a problem lurking within the pycurl
> >> > easy handle's reset function.  If I reset an easy handle and then
> >> > attempt to use it again, the code fails in weird ways.  In the
> >> > particular case that I came across, calling multi.info_read() on a
> >> > handle that has been reset leads to the following stack trace:
> >> >
> >> > $ ./reset.py http://www.google.com
> >> > Recd 5665 bytes from http://www.google.com
> >> >
> >> > Traceback (most recent call last):
> >> >  File "./reset.py", line 65, in ?
> >> >    main()
> >> >  File "./reset.py", line 37, in main
> >> >    count, good, bad = cm.info_read()
> >> > pycurl.error: (0, 'Unable to fetch curl handle from curl object')
> >> >
> >> > I'll attaching the test case.  It illustrates the problem with using
> >> > reset().  If you uncomment the line that resets the easy handle, the
> >> > program runs in a loop.  Otherwise, it dies with the above stack after
> >> > it has been reset.
> >> >
> >> > The problem seems to be due to the fact that when a handle is reset,
> >> > it's not re-configured the way a new handle is in pycurl.c do_curl_new().
> >> >
> >> > do_curl_reset() looks like this:
> >> >
> >> >        static PyObject*
> >> >        do_curl_reset(CurlObject *self)
> >> >        {
> >> >            unsigned int i;
> >> >
> >> >            curl_easy_reset(self->handle);
> >> >
> >> >            /* Decref callbacks and file handles */
> >> >            util_curl_xdecref(self, 4 | 8, self->handle);
> >> >
> >> >            /* Free all variables allocated by setopt */
> >> >        #undef SFREE
> >> >        #define SFREE(v)   if ((v) != NULL) (curl_formfree(v), (v) = NULL)
> >> >            SFREE(self->httppost);
> >> >        #undef SFREE
> >> >        #define SFREE(v)   if ((v) != NULL) (curl_slist_free_all(v), (v) = NULL)
> >> >            SFREE(self->httpheader);
> >> >            SFREE(self->http200aliases);
> >> >            SFREE(self->quote);
> >> >            SFREE(self->postquote);
> >> >            SFREE(self->prequote);
> >> >        #undef SFREE
> >> >
> >> >            /* Last, free the options */
> >> >            for (i = 0; i < OPTIONS_SIZE; i++) {
> >> >                if (self->options[i] != NULL) {
> >> >                    free(self->options[i]);
> >> >                    self->options[i] = NULL;
> >> >                }
> >> >            }
> >> >
> >> >            return Py_None;
> >> >        }
> >> >
> >> >
> >> > It calls curl_easy_reset(), which resets the options configured on the
> >> > easy handle.  It then invokes util_curl_xdecref with flags 4 | 8, that
> >> > clear up some internal state maintained by pycurl.  Unfortunately, the
> >> > handle never gets re-configured.
> >> >
> >> > The do_multi_info_read() has a portion that looks like this:
> >> >
> >> >        /* Fetch the curl object that corresponds to the curl handle in the mess
> >> >        res = curl_easy_getinfo(msg->easy_handle, CURLINFO_PRIVATE, &co);
> >> >        if (res != CURLE_OK || co == NULL) {
> >> >            Py_DECREF(err_list);
> >> >            Py_DECREF(ok_list);
> >> >            CURLERROR_MSG("Unable to fetch curl handle from curl object");
> >> >        }
> >> >
> >> > Here, if CURLINFO_PRIVATE can't be found, we die with the "Unable to
> >> > fetch curl handle from curl object" message that was seen in the stack
> >> > trace.  After the call to curl_easy_reset() the value for
> >> > CURLOPT_PRIVATE has been erased.  The only routine that sets this value
> >> > is do_curl_new():
> >> >
> >> >    [pycurl.c: line 777]
> >> >
> >> >    /* Set backreference */
> >> >    res = curl_easy_setopt(self->handle, CURLOPT_PRIVATE, (char *) self);
> >> >    if (res != CURLE_OK)
> >> >        goto error;
> >> >
> >> > It looks to me that the code between lines 771 and 815 should be
> >> > extracted into a separate routine that is invoked after a successful
> >> > do_curl_new() as well as do_curl_reset().
> >> >
> >> > I'm attaching the test case as reset.py.  Any thoughts on this from the
> >> > pycurl team?
> >> >
> >> > -j
> >> >
> >> > _______________________________________________
> >> > http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-python
> >> >
> >> >
> >> _______________________________________________
> >> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-python
> >
> > _______________________________________________
> > http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-python
> >
> >
> _______________________________________________
> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-python

_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-python

Received on 2009-04-16