On Sun, 10 Jun 2007, Robert Iakobashvili wrote:

> Let's say, somebody wants to emulate behavior of Firefox or MSIE, using
> libcurl. To the best of my knowledge, the browsers are fetching a page,
> looking into the page body and are either re-using existing connection by
> pipelining or opening new connections to fetch the found images.
>
> Any your ideas regarding the "best mode" of such implementation and which
> current features or new libcurl features to be added to facilitate such
> emulation would be very much appreciated.

Ok, some simple facts first:

1) libcurl's use of connections is not designed after how browsers work or
might work, since I don't know how they do it and I don't think libcurl needs
to mimic browsers that closely.

2) Browsers typcially does not use pipelining, or at least until recent years
when Firefox added the ability to switch it on.

3) The http spec recommends using no more than two connections to a single
server, and I believe they intened the primary connection to be used to get
the main html, and then you do a second connection (with pipelining) to get
all the images etc that the main HTML refers to.

For libcurl, you would add an easy handle for each connection you want to
make. libcurl's pipelining support doesn't currently allow you to do any fine-
grained control on what gets pipelined or not, so if you enable it you'll get
pipelining for every subsequent request sent to the same server.

-- 
  Commercial curl and libcurl Technical Support: http://haxx.se/curl.html