cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Can't view strange Java URL with Curl. Is it possible?

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Tue, 13 Nov 2001 08:13:19 +0100 (MET)

On Tue, 13 Nov 2001, Nick Chirca wrote:

(I think this dicusssion belongs on the libcurl list rather than the main
list so it was moved here.)

> When downloading, a very strange space (" ") appeared in the html pages I
> downloaded. And the spece got into my targeted links/info that I needed
> to parse. After loosing some time with this problem (I couldn't extract
> the info), I was able to find a way to get that info out...
>
> But I just wanted to tell you guys that yes, it seems that curl is
> messing with the received data...

Let's not mix curl with curl here. I'm not a fan of general statements like
this.

Curl, as in the command line tool, has not been reported to be "messing with
the received data" the last couple of years. curl as in libcurl used from
lots of different languages using various APIs have not been reported to do
so either. All these versions of curl are being used extensively, every day,
by thousands of users (hundreds of them being subscribed to these mailing
lists). I don't think you can say that "curl is messing with the received
data" in these cases.

Your particular case, however, which is using the libcurl interface code for
perl seems to indicate that there are problems.

> and another problem I have to monetion here is....
>
> I was doing:
>
> $url="http:\/\/www.somesite.com\/script.pl?page="
> for ($i=1;$i<430;$i++){
> $nurl="$url$i";
> --
> get_with_curl ($nurl)
>
> }
>
> The problem is that curl crashed a few times (many times), during this
> loop.

Please! What kind of wizards do you think we are? How on earth are we gonna
find (or even correct) any problems in such an obscure piece of code?

What did the code look like? "crashed", what does that mean? What can you dig
up with a debugger?

> Fortunately, I was able to see what $i was and start over from that $i
> and continue. But after some pages were downloaded, the crash appeared
> again... and so on until I downloaded all the pages...

This may come as a surprise, but I don't spend all these hours on this
project just because I like getting a few hundreds emails per day. I honestly
try to correct problems, add features and "support" people that need it.

But I need you to do your part. I need your help to help you.

There's no point at all to tell people that "it crashes". What are we gonna
do with that information?

-- 
    Daniel Stenberg -- curl groks URLs -- http://curl.haxx.se/
Received on 2001-11-13