Hi,

Thanks for your reply.

I use curl 7.23.1. The network is local ethernet network, 4 servers connect to the same switch via 1Gb network interface card.
I use curl command line as following:
curl -L "http://hosename:port/file" > /dev/null
The file is very large, 10GB at least.
 
My requirement is just read data from a http server and each time just read the size of data what I want to read, may read a lot of times.
So I use libcurl to connect the http server with the flag CURLOPT_CONNECT_ONLY, and send http header myself, and then read data using curl_easy_recv when I want to read some data. That means I can treat a file on http server as a local sequence file and read whenever I need to read.

The problem is curl command line and libcurl are both much slower than what I expected and wget.

Any suggestion is welcomed.

Best Regards

Zhanwei Wang
 
----- Original Message -----From: Dan Fandrich <dan_at_coneharvesters.com>
To: curl-users_at_cool.haxx.se

On Fri, Jan 13, 2012 at 04:41:13PM +0800, curl wrote:
> I use libcurl in my project to get the file on remote http server.

What version of libcurl? What kind of network connection do you have to
the remote server? Are you doing straight HTTP or HTTPS? If the latter,
which SSL library are you using?

> I tried the curl command line and I found that the download speed is only
> 12MByte/sec.
> I tried the wget command line and the download speed is 110MByte/sec.

That does seem odd. The curl command-line client is generally no slouch.
What command-line are you using do do the download?

> I thought that there was something wrong with curl command line, and use
> libcurl in my project. I use curl_easy_recv to receive data and the download
> speed is the same with curl command line, 12MByte/sec.

What kind of protocol are you using? It's very seldom necessary to use
curl_easy_recv().

> For testing, I just receive the data and simply write to /dev/null, so there is
> no disk i/o on client side. I use dell server which have 12 core and 24
> physical threads, 64GByte memory to do the experiments. I don't no where the
> problem is, and is there anyone can help me ?

>>> Dan
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2012-01-13