cURL / Mailing Lists / curl-library / Single Mail


Re: [Semi-OT] Any perl tools to parse the client side HTML?

From: Nick Chirca <>
Date: Mon, 26 Nov 2001 23:57:05 +0200 (EET)

> I'm writing some perl scripts to do some automated web server
> testing. Basically, I need to read a URL, parse out the HTML
> returned to me (specifically, a form with input fields), modify
> some input fields according to my test script's values, and then
> POST the resulting request back to the server. In other words,
> I want my script to appear as a real user/browser...

You'll have to modify the HTTP headers, the user agent name, manage
cookies, redirects and stuff like this. I tried to achieve that through an
older version of libwww also known as lwp. You can find out more about it
on cpan site. I gave up using/researching on lwp after I discorvered,
installed curl/libcurl. With Daniel's help I was able to do what I was
looking for (log in to a website, manage cookies, redirects, fill in forms
and stuff like this).

> I currently have the perl HTTP package working enough to download
> the URL, and it can post as well,

Can you show/send me some code/script exemples ? I am still a beginner in
this Internet agent/crawler stuff and I could use any exemple/help.

"It's nice to be important, but it's more important to be nice !"
Received on 2001-11-26