cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: [SAD TRUTH] does curl_multi handle can be accessed from 2 threads?

From: Christian Grade <spam_at_tickles.de>
Date: Thu, 07 Sep 2006 21:13:53 +0200

My requirement is/was to have a libcurl module which consumes data
(remote paths/urls) from [two] producing threads. At the same time,
the libcurl module is/was supposed to produce data (mainly files)
for [three] consuming threads. Modelling the logics for the *data-
structure-wise separation* of the *transfer statús*, I found I'd
better adapt "multi.c" by building in some locking mechanisms and
by replacing the "multi.c" list with a *lock-free* alternative:
one thread transferring, one thread retrieving data about transfers.
I didn't get far with this.
When the necessity arose to even adapt "url.c" (monitored strings)
and as I didn't find out how to retrieve a stack of redirection urls
and when I heard the multi-interface will undergo a major overhaul
soon anyway, I took a break from fiddling with it, pondering about
alternatives.

I must have overlooked toUpper( "curlopt_private" ). Keyword 'private'
(suggesting "better keep hands off") and 'char*' (suggesting "it's a
string") didn't seem to catch my attention. Now the existence of this
option fulfills the *should-at-least-have* part.

I see three transfer scenarios (downstreaming):
[1] Module supplied with url, file name, optional offset (resumption)
[2] Module supplied with url, continuous buffer, salvation function
[3] Module supplied with url, chunked buffer, what-if/process function

This introduces new data members which one has to care for:
book-keeping. So, one could associate these with 'easy handles'
but it would be more performant, less tedious to implement if
these were in the multi-interface already. One would have two lists
for the same concept in parallel otherwise.

So, why 'unique' IDs for connections? Encapsulation and abstraction.
When the libcurl module is running, and this practically an infinite
mode, there's little need to cope with 'easy handles' themselves
outside a universal module. The number of maximum simultaneous transfers
is 'pretty' constant; naturally subject to hardware limitations. Why
would the outside have to care which handle is used for a connection?

Regards
Chr. Grade

Daniel Stenberg wrote:
> On Wed, 6 Sep 2006, Christian Grade wrote:
>
> (Please don't top-post)
>
>> You actually end up with two parallel bookkeeping mechanisms for the
>> transfer data at two code abstraction levels: one in "multi.c", one
>> in your own wrapper, where one level is redundant. This is
>> undesirable per se.
>
> What "bookkeeping mechanisms" are you talking about that you need to
> do that the multi interface already does? Or rather, in what way can
> the interface help you do that bookkeeping?
>
>> Things become even more complicated if you need to associate
>> additional data with an hEasy. An 'easy node' should have at least a
>> void* where you could pass your own struct to.
>
> You mean like CURLOPT_PRIVATE ?
>
>> On top of that, I think transfers should rather be identified by a
>> hashed key (via url string) only.
>
> Why? What would the gain be? I don't see what the difference would be
> to your app. You still need to create an entity and tell what URL to
> transfer.
> So, please elaborate with more details how that would work.
>
>> I don't understand why you don't understand my complaints.
>
> And I don't understand why you don't understand that I don't
> understand...
>
> So perhaps we can go back to this point:
>
> Can you please tell us what you're trying to do? And then perhaps, can
> you please tell us (preferably with API descriptions or a patch or so)
> how you'd like the API to be like.
>
Received on 2006-09-07