Hi,

Daniel Stenberg wrote:
>> To solve this I guess I'll need to change the speed calculation based
>> on some sliding window average and not based on the entire transfer.
>> Before I start playing around with the transfer speed calculation code
>> to achieve this, I'd like to know why a cumulative average approach
>> was chosen in the first place.
>
> I think we did it that way simply because it was simple and not too wrong.
>
> I would agree that a sliding window probably is a better approach, and
> in case you want to change the speed limitation in the middle it'd be
> more or less required.

as I've been looking into limiting a tcp connections outbound traffic
lately, I'll share my experince, as I initially had problems with
bursts, wanted to avoid unnecessary syscalls and keep the memory
overhead at a minimum.

I choose 1 second as sliding window, and started with a *real* sliding
window, wget gave me a good idea how to do better, therefore my approach
is at least similar to wget, but as wget is blocking io this might be
usefull anyway.

I used libev for polling and timers.
Timing is in milliseconds.

Before sending I calculate how much we are allowed to send,
the function used is "connection_throttle" which is attached.

Basically, connection_throttle takes the "current" time, and calculates
the timespan of the current window, called "delta".
Then we calculate how much time the max_rate "expect" us to spend
sending that data.

Then we calculate the number of bytes we can sent using delta + 125ms *
max_rate_per_second/1000.

If the expected value (expect) for the timespan to send data is larger
than the real value (delta), we already sent to much.
To avoid "microsleeps" of the form expect - delta < 200ms,
we ignore the fact that we already sent to much, but store expect-delta
as sleep_adjust for later use and sent the number of bytes we calculated
if we are allowed to sent more than 0 bytes.

If our sleep time is larger than 200ms, or we could not sent at least
one byte, we sleep, at least 200ms, adding the sleep_adjust from
previous runs.
So we suspend sending data on this connection, and install a timer which
will "wake up" the connection after 200ms+sleep_adjust to continue
sending data.

If the expected value (expect) is smaller than the used time, we can
still send data.
To send enough, we add data for the next 250ms, using
(delta+250)*max_rate/1000-already_sent_bytes_in_this_period.

All counters are reset if the last sliding window start is older than 1s.

If we sent something, we update the number of bytes we already sent.

The magic numbers, 125, 200, 250.
I wanted the packets sent to be as large as possible, avoiding header
overhead, avoiding syscalls, avoiding sysload.
Therefore we sent more than we are allowed to given to a certain
threshold, and then we sleep, suspend sending, and continue.
For sleeping, sleep as long as possible, the longer we sleep, the larger
will be the next send.
Additionally I wanted to avoid getting the current time each time it was
required, as it is another syscall.
Therefore I used a "current" time which gets updated all 25ms by the
mainloop.
Therefore these magic numbers are by try and error.

The result does not draw a straight line when looking at throughput, but
it does a fairly good job with the minimum overhead required.

Of course one can transfer data with more exactly the allowed rate, but
the smaller the packets get, the worse for the network and system.
Therefore I prefer'd the slightly less accurate and more complex version
which sends too much at a time in large packets and sleeps in between to
keep the max rate.

The code which does the job for me is attached, does not compile as it
is only an excerpt.
I added some printfs to 'clarify' things,
removed parts from the struct definitions which are not required
and commented the positions where the gettimeoftheday would be required
if we did not use the .25ms resolution,
might be a useful reference to get something like this into curl.

MfG
Markus