cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: async IO using libCURL

From: Cory Nelson <phrosty_at_gmail.com>
Date: Wed, 11 Aug 2010 04:30:57 -0700

On Tue, Aug 10, 2010 at 10:29 PM, Jogeshwar Karthik Akundi
<ajkarthik_at_gmail.com> wrote:
> On Wed, Aug 11, 2010 at 3:14 AM, Daniel Stenberg <daniel_at_haxx.se> wrote:
>>
>> On Tue, 10 Aug 2010, Jogeshwar Karthik Akundi wrote:
>>
>>> Write shouldn't block. I give data to write, CURL would give a
>>> writeComplete_callback later that write is complete.
>>> Read shouldn't block. I should get a data_callback whenever data is
>>> received
>>> on socket.
>>
>> That's not really how it works in anything but Windows perhaps[*].
>>
>> IO in Linux and unix is generally non-blocking not asynchronous, so you do
>> as much as possible immediately without blocking, and then you wait for the
>> socket to have more data or be prepared to get more data.
>>
>> If you truly want "asynchronous", you build it with threads but the
>> underlying fundamentals are still "just" non-blocking and libcurl provides
>> such non-blocking fundamentals just fine.
>>
>> I'll second Frank's question: what exactly are you planning to do that the
>> normal multi interface can't do for you?
>
> We use libCURL only to do https and to handle proxies. After the connection,
> it is all our custom data sent directly on socket. So, we use
> CURLOPT_CONNECT_ONLY option.
> My application is mainly about sending lot of data to server. But I also
> need to be reading some stuff which server keeps sending to me. Main problem
> here is that I cannot do a read while I am writing. If the socket is in a
> writable state AND I give a chunk of data to CURL, write blocks till the
> data gets written. As my application is really write-intensive, the reads
> are coming in very delayed. CURL is non-blocking but I am looking for
> asynchronous framework to do reads/writes in parallel (like ACE library- if
> I understand it correct, ACE is a true asynchronous framework).
> Also, the same codebase has to run on Windows/Linux/Mac. So prefer to stick
> to libCURL.

You don't need a fully asynchronous framework to do this efficiently.

As "non-blocking" should imply, I/O (including writes) should not
block in this mode. If it is, then I'd guess you're using it wrong
and you should post some code that replicates your issue.

The basic pattern is:
1) wait until you can perform I/O.
2) perform non-blocking I/O until it returns an error saying it would block.
3) goto 1.

In your case if a write is blocking, I would guess you messed up
somewhere in (2).

-- 
Cory Nelson
http://int64.org
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Received on 2010-08-11