cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: libcurl + libevent2: Stalling if no data is received/written / Improving hiperfifo example

From: Dirk Manske <dm_at_nonitor.de>
Date: Wed, 29 Sep 2010 21:19:11 +0200

On Monday 27 September 2010 23:47:14 Daniel Stenberg wrote:
> I'll run some tests with a patch like attached.
According to my tests all problems discussed in this threads are solved. :-)
I guess you can drop item 264 from TODO-RELEASE.

Maybe we could get some positive feedback from the orignal starter of
this thread...?

> > So applications have to service an own counter, or simply call
> > curl_multi_info_read and see if there are messages.
>
> Yes, we should fix the hiperfifo example accordingly
I'm on that. While hacking I've discovered more questionable code
and behavior, which we should clear.

1) hiperfifo was written before 7.20.0, in event_cb and timer_cb:
  do {
    rc = curl_multi_socket_action(g->multi, fd, action, &g->still_running);
  } while (rc == CURLM_CALL_MULTI_PERFORM);

While digging into the internals I released that curl_multi_socket_action
will never return CURLM_CALL_MULTI_PERFORM... later I read the man page:
  Before version 7.20.0: If you receive CURLM_CALL_MULTI_PERFORM, this
  basically means that you should call curl_multi_socket_action(3) again
  before you wait for more actions on libcurl's sockets. ...

That means "do {" and "} while (rc == CURLM_CALL_MULTI_PERFORM);" is obsolete
now and confuses users. So I've removed it.

2) Progress callback is always(?) called two times with the sames values
while receiving data. This happens while curl_multi_socket_action is
processed. To make this visible, I've added debug messages to event_cb
before and after the call.

Output from hiperfifo after fed with an URL of a famous site:

event_cb: <curl_multi_socket_action>
* HTTP 1.1 or later with persistent connection, pipelining supported
< HTTP/1.1 200 OK
< Date: Wed, 29 Sep 2010 18:20:53 GMT
...
progress_cb: http://curl.haxx.se/ (1194/11939)
progress_cb: http://curl.haxx.se/ (1194/11939)
event_cb: </curl_multi_socket_action>
event_cb: <curl_multi_socket_action>
progress_cb: http://curl.haxx.se/ (2634/11939)
progress_cb: http://curl.haxx.se/ (2634/11939)
event_cb: </curl_multi_socket_action>
event_cb: <curl_multi_socket_action>
progress_cb: http://curl.haxx.se/ (4074/11939)
progress_cb: http://curl.haxx.se/ (4074/11939)
event_cb: </curl_multi_socket_action>

... and so on, until finished.

Surely not a high priority bug.. but smells like inefficiency.

3) Calling curl_multi_remove_handle inside a curl_multi_info_read
loop?

Original code from hiperfifo.c:
  /*
    I am still uncertain whether it is safe to remove an easy handle
    from inside the curl_multi_info_read loop, so here I will search
    for completed transfers in the inner "while" loop, and then remove
    them in the outer "do-while" loop...
  */
  do {
    easy = NULL;
    while ((msg = curl_multi_info_read(g->multi, &msgs_left))) {
      if (msg->msg == CURLMSG_DONE) {
        easy=msg->easy_handle;
        res=msg->data.result;
        break;
      }
    }
    if (easy) {
        curl_easy_getinfo(easy, CURLINFO_PRIVATE, &conn);
        curl_easy_getinfo(easy, CURLINFO_EFFECTIVE_URL, &eff_url);
        fprintf(MSG_OUT, "DONE: %s => (%d) %s\n", eff_url, res, conn->error);
        curl_multi_remove_handle(g->multi, easy);
        free(conn->url);
        curl_easy_cleanup(easy);
        free(conn);
    }
  } while (easy);

Hmm, I don't know this too... so I put the remove block into the inner loop,
the result: it still works, no segfault after throwing 10 different urls at
once into the fifo.

I'll send a cleanup/improvement patch tomorrow.
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2010-09-29