On Wed, 6 May 2009, David Hounsome wrote:

> Curl (firefox and IE) "seem" to chop the message up into seperate small
> chunks, each part sperated by the hex value boundry value
> (-----------------------------7d92fd0a04d4) each chunk is sent in turn,
> after each arrives the target requests the next and then reconstructs the
> message when it has them all.

That's not a correct description.

That's a HTTP multipart formpost you describe, and there's no "target requests
the next" involved there. The client sends the entire thing as a stream.

> For some reason, my hardware would recieve the first and no more would get
> sent. I did not have time to work out why they were not being sent (if it
> was a hardware or Curl problem) so I had to find a different method of
> sending the message. LWP in perl is less clever than curl, rather than
> sending lots of small messages, it sends one huge message with all the data
> in it.

I disagree with this description too.

This data is a TCP stream. You can't sent it as one large or many small ones.
It's a series of packages sent over the wire. In most scenarios you never send
more than 1460 (HTTP) bytes in each packet.

You _can_ use larger or smaller buffers to send() when the data is sent from
the client, and I bet we can optmize how libcurl does this for multipart
formposts.

Still, this causing problems in your end is more a proof of your system's
TCP/IP (or HTTP?) handling being foobar, and if it's this broken then I
wouldn't trust it to work for other cases either simply because the timing or
the sizing of the packets differ - when all those variations are perfectly
valid TCP/IP traffic.

> So, might it be possible to add an option to curl to send large messages in
> one go rather than chopping them up?

It doesn't chop them up. see curl/lib/formdata.c:Curl_FormReader() for the
exact details how it fills the packets for sending this kind of data.

> but in the time scales I have/had this was the best i could come up with.

I fully understand time pressure and wanting to get a working thing shipped.

But when I consider things for the main code I need to consider the big
picture and the details and there's something not quite right here...

Thanks for taking your time to desrcibe it for me!

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html