RE: Error 6 / Error 23 after long time running
Date: Wed, 23 Apr 2008 12:46:17 -0700
Arthur Rodrigues wrote:
> I'm running a multi-threaded webCrawler using Curl, but eventually,
> downloading about 600 sites
> it starts return error 6 or error 23 even for existent sites. I'm
> on Gentoo 2.16.2.
> For those sites returning error, I easy access them via browser.
> Do you have any ideas of what might be happening?
Two words immediately come to mind: memory leak.
Which version of curl are you using?
Any HTTPS connections?
Can you reproduce the problem if you hit the same site instead of
crawling over 600 different ones?
Received on 2008-04-23