Hello,

There is a bug (at least I think it is a bug) in pycurl. The original
problem was described here:

https://mail.python.org/pipermail/python-list/2015-April/688592.html

Description of the problem:

When calling curl.perform() on a curl instance I get this:

pycurl.error: (55, 'select/poll returned error')

On the server side, there is no error message at all. I suppose that the
curl client tries to call select/poll on the socket. It fails and then
returns with an error.

This error happens only if the file to be POST-ed is big enough. Files
above about 2048MB cannot be sent over with pycurl.

Today I have encountered the same problem, but with different version of
pycurl:

% python3
Python 3.4.4 (default, Apr 30 2016, 01:16:56)
[GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final
208032)] on freebsd10
Type "help", "copyright", "credits" or "license" for more information.
>>> import pycurl
>>> pycurl.version
'PycURL/7.43.0 libcurl/7.48.0 OpenSSL/1.0.1l zlib/1.2.8'

The traceback is something like this:

Traceback (most recent call last):
  File "backup.py", line 137, in <module>
    res = c.send_backup(fname, fpath, progress_percent)
  File "/usr/home/globalhand3/blindbackup/client.py", line 183, in
send_backup
    return self("backup", files=[(fname, fpath)])
  File "/usr/home/globalhand3/blindbackup/client.py", line 166, in __call__
    percentfunc
  File "/usr/home/globalhand3/blindbackup/client.py", line 146, in _post
    return self._perform(c)
  File "/usr/home/globalhand3/blindbackup/client.py", line 71, in _perform
    curl.perform()
pycurl.error: (55, 'select/poll returned error')

My code uses callbacks to keep track of the progress of the POST operation:

        curl.setopt(curl.NOPROGRESS, 0)
        curl.setopt(curl.PROGRESSFUNCTION, self._progress)

It is suspicious that the traceback contains the callback (called
"percentfunc") before self._perform(c). It has been suggested, that the
perform() call returns with "interrupted system call" in the underlying
C code. I cannot go deeper into this - my knowledge of C is faded over
the years.

Happens on FreeBSD, but it works fine with Windows clients - I have sent
files >4GB from windows clients with the very same (pure python) code.

I can work out an MWE if needed.

Below there is some example code. It is not a complete working example,
but contains all info needed to reproduce the problem.

Below 2048MB it works, above 2048MB it throws error 55.

This has been a problem for over a year. Is this a real bug, or am I
doing something wrong?

Thanks,

  Laszlo

    def _post(self, getparams, postvalues, files=None, percentfunc=None):
        c =
self._curl(self.server_url+"?"+urllib.parse.urlencode(getparams),percentfunc)
        c.setopt(c. POST, 1)
        filevalues = []
        if files:
            for fname, fpath in files:
                filevalues.append((fname, (
                    c.FORM_FILENAME, os.path.split(fpath)[1],
                    c.FORM_FILE, fpath,
                    # c.FORM_CONTENTTYPE,"application/octet-stream"
                )))
        # print(postvalues+filevalues)
        c.setopt(c.HTTPPOST, postvalues+filevalues)
        return self._perform(c)
       
    @classmethod
    def _perform(cls, curl):
        e = io.BytesIO()
        curl.setopt(curl.WRITEFUNCTION, e.write)
        curl.perform()
        status = curl.getinfo(pycurl.HTTP_CODE)
        curl.close()
        if status != 200:
            e_value = e.getvalue().decode("UTF-8")
            try:
                e_value = json.loads(e_value)
            except:
                pass
            raise ReqError(status, e_value)
        else:
            return e.getvalue()

    def _curl(self, url, percentfunc):
        curl = pycurl.Curl()
        curl.setopt(pycurl.URL, url)
        curl.setopt(pycurl.USERAGENT, "BlindBackup v1.0")
        if self.server_crt:
            curl.setopt(pycurl.SSL_VERIFYPEER, 1)
            curl.setopt(pycurl.SSL_VERIFYHOST, 2)
            curl.setopt(pycurl.CAINFO, self.server_crt)
        else:
            curl.setopt(pycurl.SSL_VERIFYPEER, 0)
            curl.setopt(pycurl.SSL_VERIFYHOST, 0)

        self.percent_up, self.percent_down = 0.0, 0.0
        self.percentfunc = percentfunc
        curl.setopt(curl.NOPROGRESS, 0)
        curl.setopt(curl.PROGRESSFUNCTION, self._progress)

        return curl

    def _progress(self, download_t, download_d, upload_t, upload_d):
        if upload_t:
            percent_up = round(100.0 * upload_d / upload_t, 1)
        else:
            percent_up = 0.0
        if download_t:
            percent_down = round(100.0 * download_d / download_t, 1)
        else:
            percent_down = 0.0
        if percent_down != self.percent_down or percent_up !=
self.percent_up:
            self.percent_up = percent_up
            self.percent_down = percent_down
            if self.percentfunc:
                self.percentfunc(percent_up, percent_down)

_______________________________________________
https://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-python
Received on 2016-09-01