How to do optimal FTP upload for multiple files?

Hello everybody

I’m developing a cross-platform app that among other things uploads N files to M FTP servers (same batch of files to every FTP server). So far I have K threads where each of them in a “for" loop uploads a file from the batch (each thread creates an “easy" curl handle, configures it and does curl_easy_perform()). libcurl code in my for-loop is mostly “stolen” from official FTP upload examples (example with ftp retry).

The reason why I’m asking this is the following: I got reports from users that my app “uploads very slowly” so I started this investigation. Default upload code heavily (like x3-x4 on my machine) underperforms FileZilla upload and commandline “ftp" utility even for 1 file to 1 server (that eliminates my theoretical threading implementation bugs etc.).

If, when doing FTP upload for 1 file for 1 server, I’m setting CURLOPT_UPLOAD_BUFFERSIZE, upload speed increases for some users of my app to “almost satisfactory”. However, same setting cause FTP response timeouts on _every_ transfer for other users (who, if I understand correctly, have much lower bandwidth). On my machine (macbook pro 2015) and bandwidth I feel almost no difference when playing with UPLOAD_BUFFERSIZE.

Therefore comes a question: is there any idiomatic way of using libcurl for such cases that will solve upload speed issues for users both with big and small bandwidth?

Thank you,
Taras Kushnir

-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2019-07-16