Daniel Stenberg <daniel_at_haxx.se> wrote:
> Hello friends,
>
> Georg Lippitsch reported that he could reach twice the download
> performance with libcurl simply by upping CURL_MAX_WRITE_SIZE to 40960.
>
> As this is bound to be dependent on the TCP stack, I would be very
> interested to get to hear from others if changing CURL_MAX_WRITE_SIZE (to
> whatever value really) will change the transfer speed in any significant
> way.
>
> I figure you just run a test with libcurl compiled as before, then build
> a new version with increased CURL_MAX_WRITE_SIZE and re-run the same
> test. Do you spot any difference?

Changing CURL_MAX_WRITE_SIZE didn't help in my case. Making rcvbuf of
the listener bigger suprisingly helps a lot on my GigE home LAN with
mtu=1500.

client:
  < /dev/zero curl -T- -H 'Expect: ' http://192.168.1.31:3000/ >/dev/null

server (for sufficiently loose definition of "server"):
  socat TCP-LISTEN:3000,rcvbuf=4194304 - | \
    dd >/dev/null bs=1M count=10000000000000000

Adding the 'rcvbuf=4194304' changed throughput from 35 M/s => 75 M/s
Not much I did from the client side helped (my "server" is also a
much slower box than my client running curl).

Linux 2.6.30 on both ends[1].

> Either way (change or no change) I'm interested in hearing the results.
> What protocol, what OS/version, what buffer sizes and what the speed
> differences were!

> As Goerg's test was for a 2Mbit connection, I figure it is worth doing
> test both against something that "slow" but also against localhost or
> similar where the speeds should be much higher.

Latency is also a huge factor in sizing buffers correctly for maximizing
high bandwidth connections. And in George's case, it sounds like he's
on a high latency connection (mobile device) with a large
bandwidth-delay product (bandwith * latency). The increased
CURL_MAX_WRITE_SIZE also causes SO_SNDBUF to be increased for win32 to
something probably closer to his BDP (2 Mbit * 200 ms = 50 k).

[1] Modern Linux kernels should dynamically resize buffers to better
accomodate long fat networks; perhaps it detected my network latency
was too low to auto resize...

-- 
Eric Wong