cURL / Mailing Lists / curl-library / Single Mail

curl-library

FTP Upload Large file give CURLE_OPERATION_TIMEDOUT error

From: Thomas Chavanis <thomas.chavanis_at_alwancolor.com>
Date: Tue, 21 Sep 2010 12:23:16 +0200

Hello Everyone,

I try to create a FTP client with libcurl 7.21.1, my OS is Mac OSX 10.6.

I have encountered some problems when I try to upload large file on an FTP server (more than 60MB).

When the upload is finished (every bytes are on server) the transfer doesn't stop
and after few secondes the error 28 (CURLE_OPERATION_TIMEDOUT) occur and I have two messages
- FTP response timeout
- connection control looks dead.

I try to implement what Daniel Stenberg explain on his blog,
on the subject "Curl keeps connections alive" http://daniel.haxx.se/blog/2007/12/

But it didn't resolve my problem.

So if some one have encountered this problem and solve it I will really appreciate his/her help.

Thank you for reading and taking time to help me,

Thomas Chavanis
Software developer

PS: The code use to uplaod file:

bool ACFTPClient::putFileOnServer(QString sourceFileName, bool eraseWhenFinished)
{
    bool successOperation = false;

    if( !sourceFileName.isEmpty()){
        CURL *curl;
        CURLcode res;
        char errorBuffer[CURL_ERROR_SIZE];
        FILE *hd_src;
        struct stat file_info;
        curl_off_t fsize;
        QString localFilePathName(localFolderPath);
        QString distantFilePath = serverUrl.toString(QUrl::RemoveUserInfo | QUrl::RemovePort);

        // Add filename to the distant file path
        distantFilePath += sourceFileName;

        // The path could have been gave with or without the last /
        if( localFilePathName.at(localFilePathName.size() - 1) == '/') localFilePathName += sourceFileName;
        else localFilePathName += "/" + sourceFileName;

        std::string tempString = localFilePathName.toStdString();
        if( stat(tempString.c_str(), &file_info)){
            qDebug("Couldn't open %s", tempString.c_str());
            return successOperation;
        }
        fsize = (curl_off_t)file_info.st_size;

        // get a FILE * of the same file
        hd_src = fopen(tempString.c_str(), "rb");

        // Global init
        curl_global_init(CURL_GLOBAL_DEFAULT);

        // Easy init
        curl = curl_easy_init();
        if(curl){
            curl_easy_setopt(curl, CURLOPT_URL, distantFilePath.toUtf8().data());
            curl_easy_setopt(curl, CURLOPT_USERNAME, serverUrl.userName().toUtf8().data());
            curl_easy_setopt(curl, CURLOPT_PASSWORD, serverUrl.password().toUtf8().data());
            curl_easy_setopt(curl, CURLOPT_PORT, serverUrl.port());

            curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0L);
            curl_easy_setopt(curl, CURLOPT_PROGRESSFUNCTION, ACFTPClient::progressCallbackFunction);
            curl_easy_setopt(curl, CURLOPT_PROGRESSDATA, this);

            // we want to use our own read function
            curl_easy_setopt(curl, CURLOPT_READFUNCTION, ACFTPClient::readCallbackFunction);
            /* enable uploading */
            curl_easy_setopt(curl, CURLOPT_UPLOAD, 1L);
            /* now specify which file to upload */
            curl_easy_setopt(curl, CURLOPT_READDATA, hd_src);

            // Set a char where the error is write when there is an error
            curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, errorBuffer);

            curl_easy_setopt(curl, CURLOPT_DEBUGFUNCTION, ACFTPClient::debugCallbackFunction);

            // add sockopt function callback
            curl_easy_setopt(curl, CURLOPT_SOCKOPTFUNCTION, ACFTPClient::sockoptCallbackFunction);
            curl_easy_setopt(curl, CURLOPT_SOCKOPTDATA, this);

            /* Switch on full protocol/debug output */
            if( verbose) curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
            else curl_easy_setopt(curl, CURLOPT_VERBOSE, 0L);

            /* Set the size of the file to upload (optional). If you give a *_LARGE
                   option you MUST make sure that the type of the passed-in argument is a
                   curl_off_t. If you use CURLOPT_INFILESIZE (without _LARGE) you must
                   make sure that to pass in a type 'long' argument. */
            curl_easy_setopt(curl, CURLOPT_INFILESIZE_LARGE, (curl_off_t)fsize);

            res = curl_easy_perform(curl);

            // clean up the FTP commands list
            //curl_slist_free_all (headerlist);

            // always cleanup
            curl_easy_cleanup(curl);

            if(CURLE_OK != res) {
                // we failed
                errorInLogFile(res, errorBuffer);
                // An error occur we delete the file create on server
            }
            else successOperation = true;
        }
        if(hd_src) fclose(hd_src); // close the local file

        curl_global_cleanup();

        // Delete the source file when the user want and when the tranfer operation is a succes
        if( successOperation && eraseWhenFinished){
            QFile::remove(localFilePathName);
        }
    }
    else{
        // There are no file to transfer
    }

    return successOperation;
}

// This function is a static callback function
int ACFTPClient::sockoptCallbackFunction(void *clientp, curl_socket_t curlfd, curlsocktype purpose)
{
    int returnValue = 0;
    int activateOption = 1;
    
    returnValue = setsockopt(curlfd, SOL_SOCKET, SO_KEEPALIVE, &activateOption, sizeof(int));

    return returnValue; // 0 is return I'm sure
}

// My read callback function
// Take in the ftpupload example on the libcurl website
/* NOTE: if you want this example to work on Windows with libcurl as a
   DLL, you MUST also provide a read callback with CURLOPT_READFUNCTION.
   Failing to do so will give you a crash since a DLL may not use the
   variable's memory when passed in to it from an app like this. */
size_t ACFTPClient::readCallbackFunction(void *ptr, size_t size, size_t nmemb, void *stream)
{
    /* in real-world cases, this would probably get this data differently
     as this fread() stuff is exactly what the library already would do
     by default internally */
    size_t retcode = fread(ptr, size, nmemb, (FILE *)stream);

    qDebug("*** We read %d bytes from file\n", (int)retcode);
    return retcode;
}

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html

  • application/pkcs7-signature attachment: smime.p7s
Received on 2010-09-21