curl / Mailing Lists / curl-library / Single Mail
Buy commercial curl support from WolfSSL. We help you work out your issues, debug your libcurl applications, use the API, port to new platforms, add new features and more. With a team lead by the curl founder himself.

Re: How to do optimal FTP upload for multiple files?

From: Daniel Stenberg via curl-library <>
Date: Wed, 17 Jul 2019 00:50:34 +0200 (CEST)

On Tue, 16 Jul 2019, Taras Kushnir via curl-library wrote:

> The reason why I’m asking this is the following: I got reports from users
> that my app “uploads very slowly” so I started this investigation. Default
> upload code heavily (like x3-x4 on my machine) underperforms FileZilla
> upload and commandline “ftp" utility even for 1 file to 1 server (that
> eliminates my theoretical threading implementation bugs etc.).

I've not seen/heard this before.

What libcurl version on what platform? Have you seen this "slowness" on more
than one version/platform?

Is this against any or particular servers?

When you do these "slow" uploads, what's the RTT to server, roughly?

> If, when doing FTP upload for 1 file for 1 server, I’m setting
> CURLOPT_UPLOAD_BUFFERSIZE, upload speed increases for some users of my app
> to “almost satisfactory”.

When you set it to what value? What does "almost satisfactory" mean in terms
how it compares to other tools then?

> However, same setting cause FTP response timeouts on _every_ transfer for
> other users (who, if I understand correctly, have much lower bandwidth).

That sounds really weird and would rather imply a (separate) bug.

> On my machine (macbook pro 2015) and bandwidth I feel almost no difference
> when playing with UPLOAD_BUFFERSIZE.

In my own tests, UPLOAD_BUFFERSIZE mostly make a difference when you have near
infinite bandwidth, like uploading on localhost. (And when when using SFTP,
but I assume we're not talking about that here.)

> Therefore comes a question: is there any idiomatic way of using libcurl for
> such cases that will solve upload speed issues for users both with big and
> small bandwidth?

Increasing buffer size shouldn't hurt transfer speeds for any users!

  / | Get the best commercial curl support there is - from me
                   | Private help, bug fixes, support, ports, new features

Received on 2019-07-17