cURL / Mailing Lists / curl-library / Single Mail

curl-library

RE: HTTP PUT performance issues

From: <RBramante_at_on.com>
Date: Thu, 5 Dec 2002 23:34:23 -0500

Results are in. No diff for me.

C:\dev\curl-exp>curl --version
curl 7.10.3-pre2 (win32) libcurl/7.10.3-pre2

C:\dev\curl-exp>curl --upload-file c:/tmp/1MB.exe --header "Authorization:
Basic
 anJzY2xpZW50Ompyc2NsaWVudA==" --header "Content-Type:
application/octet-stream"

http://devjunk:8080/jrsync/servlet/JRsyncServlet/$depot/da01e.051/1/ccmagent.exe

  % Total % Received % Xferd Average Speed Time
Curr.
                                 Dload Upload Total Current Left
Speed
100 1665k 0 626 100 1664k 38 103k 0:00:16 0:00:16 0:00:00
100k

Also Mike my issue is not just the raw number performace, rather it is that
Java's Http class and a dummy toy app I wrote can push the same file to the
same server in about 1/8 the time.

Why I am seeing this and why Daniel does not (my dummy app and curl perform
the put in the same time) for him is not clear. Maybe it is windows
specific? Maybe it is some bizarre interaction between curl and this
specific webserver (tomcat)? Maybe several factors specific to my
environment?

I'm out of ideas. It would be interesting if some other people took my
dummy app and ran it side by side with curl and see what the results are.

                                                                                                                              
                      Mike Tardif
                      <mike.tardif_at_adobe.com> To: "'curl-library_at_lists.sourceforge.net'"
                      Sent by: <curl-library_at_lists.sourceforge.net>
                      curl-library-admin_at_lists.sour cc:
                      ceforge.net Subject: RE: HTTP PUT performance issues
                                                                                                                              
                                                                                                                              
                      12/05/2002 02:51 PM
                      Please respond to
                      curl-library
                                                                                                                              
                                                                                                                              

Daniel

I wonder if RBramante Apache server's PUT capability is provided though a
PUT cgi script like mine is.
If so, he may also be measuring the time of execution of that PUT cgi,
which
amount to a double copy:
 http client ===> http server ===> put cgi
of about a gig of data. Wouldn't this account for roughly halving his
performance.

Cheers,

Mike P. Tardif "UNIX is basically a simple operating
system,
Adobe Systems Canada but you have to be a genius to understand
Ottawa, Ontario K1Z 8R7 its simplicity." Dennis Ritchie.
Vox: +1 613-233-2256 x5649
Fax: +1 613-594-8886

-----Original Message-----
From: Daniel Stenberg [mailto:daniel_at_haxx.se]
Sent: Thursday, December 05, 2002 02:03 PM
To: libcurl Mailing list
Subject: Re: HTTP PUT performance issues

On Thu, 5 Dec 2002 RBramante_at_on.com wrote:

> OK sounds good. I hope you find something because I am really confused
by
> this as well if the ftp and http xfer code is the same.

Did the tests and I've learned some new things. I'll show you:

Test machine is a 733MHz Pentium III running Linux kernel 2.4.18.

All tests were done on a file I have that is 843309644 bytes big.

'httpput' sent it all in 135 seconds to my Apache2 on localhost (the
receiver
is a perl script that sends the received data to /dev/null). It makes
6246738
bytes per second. That makes it 5.95MB/sec.

Clearly my Apache receives HTTP PUT *much* slower than my wu-ftpd receives
ftp upload.

I modified my curl to always loop until EWOULDBLOCK before going back to
select(), and then uploading (HTTP PUT) the exact same.

It took 134 seconds. Pretty much the same speed, I'd say.

I then went back and fired away an older curl I have installed (7.9.9-pre1)
and it ran the same test at 136 seconds.

The amount of select()s clearly has no particular impact on this test here.
It might be because the server is the one slowing down the operation or,
perhaps more likely, it is because windows has a higher penalty for using
select().

Test number two:

I run 'nc -l -p 8888 >/dev/null' to just accept a PUT and read it at
highest
possible speed. The timings showed:

Modified curl: 44 seconds
httpput: 43 seconds
Old curl: 44 seconds

Summary:

There is no speed issues with HTTP PUT with curl on Linux. At least not
that
I can provoke to appear.

Do you think you could get the latest sources off CVS and try them on
win32?
My changes might make a different impact on Windows than on Linux.

--
 Daniel Stenberg -- curl, cURL, Curl, CURL. Groks URLs.
-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
Received on 2002-12-06