cURL / Mailing Lists / curl-library / Single Mail


Re: writing bot with libcurl

From: Lars Nilsson <>
Date: Tue, 13 Jul 2004 23:13:59 -0400

On Tue, 13 Jul 2004 19:22:29 -0700, Ryan <> wrote:
> What I want to do is write a program to log into a site and parse some
> html pages so that I can import the data into a database, will libcurl
> allow me to do this easily?

Yes, very easily (in my opinion). It will be up to you to maintain
your state and parsing the pages (libcurl for the most part only gives
you the content). I believe I've written between 15 and 20 different
programs for similar purposes, ftp and http alike, and each one does
usually not take more than a day to complete. Most of the time will be
spent figuring out what to do with the data, not how to make libcurl
give it to you. Personally, I use libxml ( for
my HTML / XML parsning needs in conjunction with libcurl.

Lars Nilsson
Received on 2004-07-14