cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Hashing while downloading

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Tue, 20 Jan 2015 00:23:20 +0100 (CET)

On Mon, 19 Jan 2015, Leon Winter wrote:

> a friend of mine is working on "modernizing" Apt of debian. Apparently there
> were running some obscure hand-written HTTP code resulting in very low
> performance. Not to mention potential security issues and lacking features.
> He therefore now rewrites Apt to use curl.

Cool!

> in order to do this one needs to link against a library providing a hash
> function. Curl however already has such dependencies and even has a small
> abstraction layer for Md5. However this is not exported

Right. I'm generally very careful with adding new APIs, especially such that
aren't strictly transfer-related and I would say MD5 isn't about transfers.

All new functions take their share of added maintenance and work.

> It is noteworthy that this copy'n'pasting already happened inside curl
> to some extend:
>
> lib/md5.c
> src/tool_metalink.c (albeit abstracting over more hash functions)

Well yes, but those two are in the library and in the tool, pretty much for
the same reason you bring up here!

> While looking into this I also noticed that the metalink code does the
> verification _after_ the download, which Daniel also mentions [0]. In the
> mentioned RFCs about the headers and XML format I found no mention of the
> time of the hash processing. Why not do it while downloading?

I don't think there's any good reason other than it hasn't been done. Possibly
because nobody has cared enough to actually do the work.

> Should we either export the awesome abstractions curl offers for hashes or
> possibly also TLS (the VTLS layer) to outside?

I would prefer to have the entire VTLS part of libcurl turned into a library
of its own that libcurl could use (although it hasn't happen because there's
just not enough desire from anywhere to drive such a change). I don't think it
is libcurl's job to offer neither crypto nor hashing functionality outside of
transfers.

> Shouldn't the metalink implementation make use of the MD5 abstraction
> already in place?

The Metalink code is not in libcurl.

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Received on 2015-01-20