curl-and-php
Re: Rotots protocol
From: Daniel Stenberg <daniel_at_haxx.se>
Date: Tue, 25 Apr 2006 08:17:12 +0200 (CEST)
Date: Tue, 25 Apr 2006 08:17:12 +0200 (CEST)
On Tue, 25 Apr 2006, vivekanandan wrote:
> how can we find a site which is using robots protocol or not..
Are you referring to the robots.txt file?
-- Commercial curl and libcurl Technical Support: http://haxx.se/curl.html _______________________________________________ http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-phpReceived on 2006-04-25