Could http pipelining be an issue ?
Date: Tue, 8 May 2007 15:19:38 +0200
Im using libcurl to download many little files permanently. For example my
application can download hundreds of ~6KB files in only few seconds. We
noticed that on local network (ping: 1ms), we have very good performances,
but from outside (ping: 35-40ms), files are downloaded very slowly. We made
some tests, and we conclued that the problem is not the bandwith, but the
Im using libcurl in that way: i have 20 threads, and each thread manages an
easy curl handle. Then, threads fight to obtain an url (in a shared pool) to
download. Most of my requests are ranged requests.
So now my question is :
Should i manage myself to ask for several objects in one request using a
ranged request like => 0-42,142-242,330-369. It means a local pool for each
thread, waiting for a minimum number of files to download. Or should i use
http pipelining ? My main problem is to dodge the latency, so make the less
number of requests.
Thank you in advance :)
-- Nicolas Remia http://www.magrathea-engine.orgReceived on 2007-05-08