On Wed, 27 Jan 2010, Pavel Raiskup wrote:

> First of all I'd like to ask you if there is a big problem with source size
> (binary size/number of source lines) or code complexity.

In general, no. But all added code adds to the amount of code we need to read
through to graps stuff so we should always strive to minimize it. The more
important detail is that the code should be written in a clear and concise way
with enough comments so that it becomes easy enough to read.

> I know, that it is possible improve library without recoursive and
> "ftp://site/*/*/*.c" idea, but it could be much more useable for app
> programmer..

Possibly, but with recursion you also get more features. Perhaps an app wants
different patterns in different subdirs? Perhaps the app wants to limit the
depth of the tree traversal? Perhaps the app wants to exclude a specific
single dir from the traversal?

If we can't offer these features, we may end up in a situation where all apps
that want recursion STILL implements it themselves simply because our way is
not flexible enough and then we have an implementation that adds complexity
for no additional benefit.

I'm not saying it has to become like this, I'm only saying we need to think
things through so that it doesn't.

> Next I really considering to parse LIST response. Even if we discard the
> recoursive idea (but I'd like rather do this), we have to let the app
> programmer know the matched filename is directory -- to allow him to create
> new curl handle and start with wildcard downloading in subdirectory.

Yes, I think you're right. It will make it less good if it treats everything
as files and thus fails the transfer on all entries that are actually
directories (or symlinks?) and not actual files.

I guess it will also make it a better function if it knows the total size of
all the files it will download, and the number of files so that the app will
know beforehand.

> And the multiple-calling curl_easy_perform idea -- it could be a little
> inconsistent behavior from libcurl and I think it is not really necessary.

Inconsistent, yes. But so is calling the write function with contents from
several files, so no matter which method we pick we're breaking new grounds
and break previous assumptions and paradigms at least slightly.

> int wildcard_callback(char *matchedstring, FILE *f, int filetype)

As Kamil Dudka brought up: FILE *f is really not the right type to pass to a
callback. It should be a 'void *' and the callback itself will then treat it
as whatever it knows was passed in.

I also miss total number of files and total size of all files (which you
mentioned further down in your mail but I didn't understand if you have
decided not to include them or not). What is 'matchedstring' ? Is that the
directory entry name?

I also like a differe error code usage. Just using TRUE/FALSE tends to end up
a mistake. Use 1 and 0 instead explictly, and everything else is reserved for
future use. That allows for a future libcurl to introduce new return codes
without breaking old apps.

> Question is where to close "FILE *f" when it is changing.

The wildcard callback is called before each transfer so it would basically say
"here comes a new file". That means the previous file ended and it would have
to be closed, or similar. A bit awkward I'd say (especially for the last
downloaded file). Possibly we should consider one callback before and one
after each file.

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html