c:\curl>vreme.bat vreme.bat c:\curl>curl -f -x myproxy:8080 -v -d "username=xxx&password=yyy&submit=Submit" -L -b cookiejar.txt -c cookiejar.txt -A "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)" http://www.vreme.com/account/login_action.php 1>toc.html * Connected to a.b.c.d (w.x.y.z) > POST http://www.vreme.com/account/login_action.php HTTP/1.1 User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0) Host: www.vreme.com Pragma: no-cache Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */* Content-Length: 65 Content-Type: application/x-www-form-urlencoded username=xxx&password=yyy&submit=Submit % Total % Received % Xferd Average Speed Time Curr. Dload Upload Total Current Left Speed 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0 * Closing live connection (#0) * Follows Location: to new URL: 'http://www.vreme.com/' * Disables POST, goes with GET * Connected to a.b.c.d (w.x.y.z) > GET http://www.vreme.com/ HTTP/1.1 User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0) Host: www.vreme.com Pragma: no-cache Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */* Cookie: PHPSESSID=4a048dfcc3abd5e064e33d61748a5c1c 0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- 0:00:03 --:--:-- 0 30 13091 30 3954 0 0 910 0 0:00:14 0:00:04 0:00:10 988 100 13091 100 13091 0 0 2950 0 0:00:04 0:00:04 0:00:00 3272 * Connection (#0) left alive * Closing live connection (#0) c:\curl>gawk -f vreme.awk toc.html 1>vreme_go_get.bat c:\curl>vreme_go_get c:\curl>curl -v -x myproxy:8080 -L -b cookiejar.txt -o "file_#1.html" -c cookiejar.txt -A "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)" "http://www.vreme.com/cms/view.php?print=yes&id=[301581-301626]" [1/46]: http://www.vreme.com/cms/view.php?print=yes&id=301581 --> file_301581.html * Connected to a.b.c.d (w.x.y.z) > GET http://www.vreme.com/cms/view.php?print=yes&id=301581 HTTP/1.1 User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0) Host: www.vreme.com Pragma: no-cache Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */* % Total % Received % Xferd Average Speed Time Curr. Dload Upload Total Current Left Speed 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- 0:00:03 --:--:-- 0 100 2344 100 2344 0 0 619 0 0:00:03 0:00:03 0:00:00 781 100 2344 100 2344 0 0 619 0 0:00:03 0:00:03 0:00:00 586 [2/46]: http://www.vreme.com/cms/view.php?print=yes&id=301582 --> file_301582.html ^CTerminate batch job (Y/N)? y