> Hi - I'm hoping that someone might have wrestled with this issue before.
> Thanks in advance for any insight.

(First of all, please don't send HTML-formatted mails to this list)

Well, since you didn't mention what site you are trying to access, I
can't really tell... :-)

> But, when I try the more complicated query on the actual site I want to
access, I get
> nothing at all returned: no error page - the connection just seems to
terminate.

Strange. Looks like a kind of filter.

> Because the actual url-encoded query text is soooooo long (1027 bytes if I
recall), I'm
> testing this by downloading their HTML form (it has session id stuff and
other dynamic fields),
> changing the action tag to a mailto addressed to me; that gets me an email
attachment with all
> my variables url-encoded. When I use curl with the -d switch and the file
name reference, I
> see the query get entered correctly, then I'm back at the DOS prompt. The
status bar shows 0
> bytes saved in my output file.

What version of curl are you using, and on which platform? You mentioned
DOS, so I assume you're on a Windows machine?

What can you see when using the --verbose option?

> Switching the form action to mailto to generate a test query worked fine
on other sites...
> just not the one I need; maybe there's something else about trying this
that's causing the
> problem. But any other approach gets me a "we cannot process your
request" response.
> Is it possible that their server is booting me because it knows it isn't
be accessed by
> a browser?

That's possible. But all it can do is to check the Agent string, the Referer
and possibly
some cookies.

> This is a subscription site, so I wouldn't be surprised if they do more
> checking than most sites (I think access to the search pages is blocked to
non-subscribing
> IP addresses - there isn't a username/password to enter.)

OK. This is interesting. You don't have to supply an id/password and you
still get to
use the service? There might be some persistent cookies involved. One other
thing that
comes to mind is if you have some kind of plugin installed in your browser
that does
the identification. But I assume you would know, since you probably are the
subscriber.

> I've tried experimenting with the -A and -e options; they don't seem to
help.
> Are there other things I can try to make my script look and act more like
a browser?
> Are there ways to set environmental variables that might be checked by the
server-side
> script?

You can affect (almost) everything sent to the server.

> Any insight on what's happening in the black-box would be appreciated -
it's hard to
> know what to try next when there's no response from the remote server at
all.

I think that you should use a network sniffer to log the requests from your
browser and
see exactly what is going on, and then try to emulate it.

> Thanks for your time.

You're welcome.

/Linus
Received on 2001-07-16