cURL / Mailing Lists / curl-users / Single Mail

curl-users

OS2 get multiple urls doesn't work

From: <datablitz_at_optusnet.com.au>
Date: Thu, 14 Sep 2000 15:36:05 -0400

Hi there

When I try to get multiple urls (ie. using []s) curl fails to retrieve
them. Single urls work perfectly. I have tried this with different sites
etc, but have never got it to operate.

Here is an actual attempt at getting multiple urls, followed by getting a
single url. This is run as a REXX script but the same result occurs when
using the command line, minus the apostrophes. rfc3.html came in fine, 1 &
2 see results below.

/**/
'curl http://www.cis.ohio-state.edu/htbin/rfc/rfc[1-2].html -o rfc#1.html
-v' 'curl http://www.cis.ohio-state.edu/htbin/rfc/rfc3.html -o rfc3.html
-v'

Ouput from curl:-

[1/2]: http://www.cis.ohio-state.edu/htbin/rfc/rfc1.html --> rfc1.html *
Connected to www.cis.ohio-state.edu (164.107.115.3)
> GET /htbin/rfc/rfc1.html HTTP/1.0
User-Agent: curl/6.5.2 (i386-pc-os2_emx) libcurl 6.5.2
Host: www.cis.ohio-state.edu
Pragma: no-cache
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */* >
  % Total % Received % Xferd Average Speed Time
Curr.
                                 Dload Upload Total Current Left
Speed
  0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:--
 0100 149 0 149 0 0 40 0 --:--:-- 0:00:03 --:--:--
 149100 149 0 149 0 0 40 0 --:--:-- 0:00:03
--:--:-- 74 * 149 bytes transfered in 2.180 seconds (68 bytes/sec).

[2/2]: http://www.cis.ohio-state.edu/htbin/rfc/rfc2.html --> rfc2.html *
Connected to www.cis.ohio-state.edu (164.107.115.3)
> GET /htbin/rfc/rfc2.html HTTP/1.0
User-Agent: curl/6.5.2 (i386-pc-os2_emx) libcurl 6.5.2
Host: www.cis.ohio-state.edu
Pragma: no-cache
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */* >
  % Total % Received % Xferd Average Speed Time
Curr.
                                 Dload Upload Total Current Left
Speed
  0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:--
 0 0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:--
 0100 149 0 149 0 0 44 0 --:--:-- 0:00:03 --:--:--
 0100 149 0 149 0 0 44 0 --:--:-- 0:00:03 --:--:--
 0 * 149 bytes transfered in 2.570 seconds (58 bytes/sec).
* Connected to www.cis.ohio-state.edu (164.107.115.3)
> GET /htbin/rfc/rfc3.html HTTP/1.0
User-Agent: curl/6.5.2 (i386-pc-os2_emx) libcurl 6.5.2
Host: www.cis.ohio-state.edu
Pragma: no-cache
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */* >
  % Total % Received % Xferd Average Speed Time
Curr.
                                 Dload Upload Total Current Left
Speed
  0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:--
 0100 1317 0 1317 0 0 305 0 --:--:-- 0:00:04 --:--:--
1317100 2645 0 2645 0 0 605 0 --:--:-- 0:00:04
--:--:-- 1322 * 2645 bytes transfered in 3.150 seconds (840 bytes/sec).

RFC1.HTML as it appeared on my system:-

<TITLE>Lookup Error</TITLE>
<H1>Lookup Error</H1>
Can't retrieve your request -
'/export/1/ftp/mirror/internic/rfc/rfc0001,rfc,0001,rfc1*' not found

Further notes:-

I am using OS2 Warp 3 Connect incl. TCPIP, emxrt 0.9d, OS2 ver of cURL
binary, hpfs drive. The output from cURL was fed into a console so I
could record it all, hence the funny looking progress indicator. For a
single url, transfer speed is normally over 3000 bytes/sec. Note the slow
speed after trying a multiple transfer.

Maybe you can help me with what I was looking for in the RFCs. Is there
any way to get a list of all urls in a directory on a 'standard' website.

Cheers Wayne

-- 
-----------------------------------------------------------
datablitz_at_optusnet.com.au
-----------------------------------------------------------
Received on 2000-09-14