Bugs item #1180358, was opened at 2005-04-10 15:00
Message generated for change (Comment added) made by nobody
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=1180358&group_id=976

Category: http
Group: bad behaviour
Status: Open
Resolution: Works For Me
Priority: 5
Submitted By: Nobody/Anonymous (nobody)
Assigned to: Daniel Stenberg (bagder)
Summary: POST limits

Initial Comment:
Version: 7.13.2
Linux/Fedore Core 2/kernel 2.6.11

POST only takes limited data (or limited word sizes) - more so than earlier versions?
(http and https)

As I recall, earlier versions could post data of thousands of bytes in length. A file containing:

__VIEWSTATE=7rZJ7Mj8uQSGbyixlxqBU%2BhHCCdr5d%2BriNWfzXmF5VaAGAgK8BBFHomckAObOmGu%2FreDoHaO2uR7bFJxzGUPmAgj%2FI6mFEAONkjbUKfKGBYF6J5lBTcOn7dAFiSf
UbtJSMMDuyGRvJ9r98VZUQtBdLXdT98HTQTMtfEsp%2FNVh8qWAGn0XI%2F%2FjzOtEuY%2BHBrYc1F%2BuOP1vHxxTovRyRT61wLuWCbA5aHnYbk%2FlBgSa4dJ0RfVva9oSWDdd5VFlZcs
FZcy7npLnZKey4O7y%2Fq7IpvguSwM%2BWTuDCgtBxIkd%2FQuNCqoPSrHFw0hlSSLr60UIdWF46D4kznAWaFMpre%2F0BWLUVYbwkCRpPGmb6I5f9EV2d%2FqHPDaWnEg8qPzmKbX%2B3Ae
vELdjj6GXtJzwg9G61RG4ZQi2rWwsMSwCyVUsLJtOqmNciahaX12v70UHTeCKshaZP5d2YmypZk%2B2fDg2cB7GJCqaCLhTsc%2FVq470N1jYQRxvzA4%2Fn6Pz1jp3g0A6GE5VlMmE6MsKJ
kM%2FJ6G6EZt7jvL9QROnx6x%2BQZdlFGZ0Y1HjqkoMup7VQ6C8d0WeUVSkvS7OESoHG8SVQSdxZJ%2B5u7hIHxGeMCc6evQEB8o%2BeeGKKBuK%2BW3aFTpGS51u0M5Pfefl4gECUP6lehY
mwGk%2FnfLpABfOJKAQMmrUanO%2FKSOdwX9IxI6mP%2BnP7G9z3FQMb7F6E63leZ%2BHudUu4vsjuGM89AAuQiJVbOY5BPO3F3rn3nKnJxvsqFEG3rLyILEEAb0ZKIKKnUkANIPhkwcyW12
vwEhIHlSJY%2F9pHc1DPgjOsnbtszv48XaCr2JMmgPcuC3A0uhxjyrGf5QHmW6sapdUpopro4HwhSxeojzEQ9l2AqHrqHANQKjaK3JgSXfKnzL6Uw08%2Byb2NVemHpBGqkm2V8uxydbWfH0
ipXjDBYughyz2B6GyjzJANaiIUdUXYXyCZTK0jisqyGg5gzRmlsUeTlVko9qrYRpvL3XSwN7rjtcnQ%2FF4867p9J8U5SL%2FGezxW2S5SuhTMsdV9ogVFoyywJUG9Tgj5LdznysjUTItUs%
2Bw6E%2FyF%2BqOrLwgiRiS6iUoxnb1bIhIWTZpPXx4p1TLZ3YdNq0n6c7490%2FZ3ZCDoUyE%2BLpixmfQlsg04Hs%2B%2Bnspgxe6ja%2FAPXFouueq1csC%2BTCCa%2FVV9FApu%2BCpc
4rN11pByAo3qUyetZnAZuKuJjpqNoDaGDEwxITD7erRSWhwad89%2BB%2B%2FWcOdR24jpQLiyc9oz2OcxyBtEw3polKwj1Dn6UTX0drBnMOZtERfRr8antVw%2Fde8ywzlwxCJ0tIiVCsRL
watIy6mu30PaK%2BAvsrjEGJkxS4PSfIn07S9VugTq7PnvwI1%2FXvD5J4XcwL%2FnWENGCQz1%2BrxX%2BGSUGglJR5k6KmxFACfZe7wV854c5d7eCWnVGljqf%2FWcxZvQLAgnXoKmP%2F
ehHV1WsDLUtzk4s3J2WOfrBLe%2BegZqn7%2FxVy4UdMYeWB%2BvUY%2B8aovSkXCuPsqeyg6FTy%2Bwjt7%2BkUETpOqgguSTirZv5zOS9r0WHJMFO9DTHYAIjZ9%2FQ%2FTZ8amSSJW50J
BV6GAgW2eje%2FxOifa%2F%2Fx0A4YQKUEDT%2BNJURDsGmihRsQ0thNKJJMnfafBz3Apwq6CtuWxKtjvRi6i7Wyv0LJu4a0bCP%2FNrAwC2ZtQBfETBH7qHSFT%2FXwHCbL8c%2B3iolQjz
l0JMyTsLn2AeR%2FWbNxlZNDxqDRTQ%2FGAlp1lmk%2F8WyqveFiqrvAzScHdV%2Fv2LDnRB%2F%2FiQiAfNQziG%2FlyYPAJptx%2FFgmBJM7J6m3KT1Pg8zZTsUhAt4if6JmtGZnT1xsfF
Ze9fNCS15EpGScWyQuf3%2FAmkgOmQe%2F62UcfP6IhMUoxesnytVQzr0rZg78gZOCmjDnmCvqRvhOtUoRn%2F8e0HFtGboUTwIaPJBkqDHr7EICl6VQQ1la%2FKInZn6jj5vRq8DiZGgBkB
cWBoQUEFiTQkpocRQj4Miqp7SS%2BnN3LW9XLSWrwyzymLtsvxoO4WLugVcaIFJtiPgoOqxZTQHAXYjLJn19gKFZz0rsMZ8i1bCjEnPUpzYsBsoKkzwgraRZXA%2FJo11XcPjU5kN4Mp1MI%
2F1QTnXk%2BLY2apHnyRV6i6mID2NgAapvGLwbq9IE8HoKJzeC5dVHltn9CiyTQ2ZqSs80JAtxS%2BZOWMNEi1TFKUUhwiWML2CoyVR9UHosHicDcT3n6aZmX36AS3qLJriN7qfEcfXs3hbt
YiMyuR5kq4V1hXLy1IbVMZa%2B8p245123nNLQdjcR4wZcOGYyCeANaGO0qnHcIMhlqX9QP1b3BK1qeRc1nJPN976ZaySVEa8fMuKyZvD72EB71hBtXXL5Jx4%2F7IoRpq1wwL%2F3NrZdE5
eiiAG4gHQaRPXTTO%2Bldl9Z9eACJveYevDCnA0CRGNVRs0VeVbPYdAPU18z6uBCu11jscph%2BXH4fw90Putq088JTe7aVuq%2FRdyR4dunjQneAsGKn39d4JBy0HhalGhxP%2FOOMJkiOx
oma%2FFJ8Ojk9a1Dv2ss3uQmaGG64Nd%2BQNYBGeSpO9nPESW6TdOPKjGaAqja3XDrvYfxkP7AdEnUiGpShzBAV4U%2FtkEJqnzTnz65uiipK59pTWAwyhJWQKpawKZ25dhtO6oBpNfS%2B8
U%2F4KasV2LDTpvpWkkn%2Fa6kHX97vB9EAU0Rnqr5ICgh6hectryzcKaSHlJwIb0qY4HV%2FYIGUZ%2BISoVgD9KwUITq2Fa8njhQ%2FEgGe6qfv9T6VasUpfURH4V5YeBWRfCNFIOFP0YX
702FNFRj5U1v%2BSlQ%2FeAfLWrtWs%2BRlityXuPpPPinQsbWG%2Fwtuxa4lH%2F4gIt%2ByI4M39MOaUj%2B3pWRvFPsFo6rtNVqK%2BuceulRbdHussRRj8euzJUv5EsbnoF94whL%2FA
TomN97UgSVRcvI1fuUv5pVw5HA4O2CGV3o5O626s7E8D1VctCLB2J64Rp0WyAxqDvPeVvjvrCqutDkXArlYTgzi2JMIskUcRf09UEzvW%2Fgq7HW7fiK0seVXyFBchX0F40o5xQ9m7ggynC8
8p51Gyg4vAtSdDDgMkWVVPY7s5BntAPbL41BQ82KQ632vl8X%2BYgtq8ThP9DYd%2BA4%2BCGGhqvpBTL%2BVVmEm20lF%2BYX0sz%2Fd4sXWRo1uO91gSyBmapHQVP2m%2F%2BCIR%2Fcf6
fhP3LsK8%2Ff3YMj5ezjTFepVEY9H8uqxFNPVzke%2FmdVrxQCE%2FWMHAArCoLeN66rgO13KVCqj%2Fgls6kv%2B8amqfM4b%2By%2FXR4bL6MNkgEnHHt0ROT1l4K0%2FigBDEodypFzsc
8W3dmhdo%2BLFJt%2BWbZ2sSyGYJCfVOpi7NP2N51Zv%2B5xJnneiUkXDHvaAiBbdBCLESVXIvdkSRH2X%2Fv%2BIdlNG4msinHy11y1C6KaAUtnoEc7EowfTwM3CnIqCMbDufC5jBHsClgJ
3V41bBv1WNTIte4NTyNj4UkvKLstjHLbv6frAc%2FUvhdw0Sw4F5FTYUHMBpp3bbwnQJuS8%2B4tAgFJkUI4fPER7RsDQjmmOh1Uyy1zDnehnYHpPPpSV1TdVlGJOk3gSeLzHaQ3cbp5pphB
xmTrAqZ9ktoU3x2gcoIIiCi0M5HTvu%2BNzhMhGtq9fgldLvHco8jC1n%2FUPd8a0f6D8uvMgBaSRzLCE0LwQzy%2FmUW8DZQw59ddzkQbBv8W9a2yaK5JNrlhNiqXYPbGxpM8JQi0oP0ybm
0ohOVpMuYuoUkhLe0xAMY29tB1nyGjWbiiMb1TX7k1TbJzCcKyD0TS2DPRPGw67o0jHErVJ7%2FhbPB2dW6DS9VZAhNU3qrnG2emoYqmE56HnSl7tGOs8dcgsVvGIIC7Eia07LOhcLwC4A4P
58AJpZ3mKmy6%2FvQHps9o6X7TdWPek5BwiTI4TjEww1qyFMG%2BBwotH7nlzIcvV27awsVuFd8NVM6NLrg1j7FJaSNYbByN43d55meNgAtXBkSuCF26TJBUhkvlwHKRCZsPXR9FAKtebMB%
2BM6M8XBmoZ7bc6sF6l4VIwZWWIf7m8cmVGx7mLvjpzxKJevoBWYyij1fy%2Fccg%2FDnKh5UJFuMS0UzDSDWYvSGjjveWVbx3uGKoKHgTkdiDgznPD%2BJ8raj1eWwQPCTjxVA4mgKLno6i
EKPKDYThYCYtttD5aLL0vkLg9kDVC1OlGQT5HtGYVPXFgIK91MHrXqvkCQFCK1gm%2FqMkb0xW8GZkaArnsVcsAiLDYpeKmNB1nn0Luv%2FKn7aVn7g142pu3cDqnYISiw4Nx8pVaGTC7%2F
4zIv4smbWZHe1lIZsfF%2Fy4apiRROquBt2SZHk9LBA7cNKC7v%2Fig65zak0gMHbW0Iu9F0e%2BEBLnX83cif2zjtIAAFyeEVHfKmIzcg7QPa61THc2GhGEjDiz4JPGUZQ4hOjJzvdDfBC9
Jex72dZfVgEmR7boSQlj36ZFWWZgTuRjgRvqP9i73wmS%2FqQCcE8VxTIocmNJpfqWABAZjb5ot59xwFoh%2FZ%2BPuh%2FoJoKP1rfO%2BtXEMqwQwtMLXM0%2Bwj4iJxzM9Z4ZHsBPKqWM
pzmfLS7ZM3FiN6dOuyJesP1SM4cgp%2BBYt7c3pFFheyySrz0OBhwCOcNGEDxP6ea%2FbD2WF1COVczepr%2BQ3F3Oh%2BKphJiMWy9d6CpDVLOJ1L4hI1a4ocDlSnbX8bYVgtTu8mIfMmgX
fy0VrmVYdDA28IM5oO7Cvu%2Bbm%2BH2%2B0xf56Q1YsdYc9Fk5Aakx3h%2BVNSpbo8xW%2FJ4LwSMrI8dzWXPkw%2BP0qk2joZcWAKqz0IwEqdOUhm%2BXUE%3D&UserId=userxx12&Pas
sword=passxxx123&LoginButton=Login

(called DAT1)

does not get posted to the phisher's URL (hopefully down, now) usingL

curl -L -b cook -c cook -d "@DAT1" "https://onlinebanking.bankofoklaoma.com/aspnet_client/system_web/1_1_4322/_fOnlineBanking_fDefault.aspx.php"

(nor does it get posted using "http").

It does get posted if I drastically cut the length of the VIEWSTATE variable.

If I recall correcly, earlier versions could make POSTs with more 1 or 2K of data.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2005-04-10 18:30

Message:
Logged In: NO

OK... I think I see the problem ... your question about the web server cleared it up a bit.

The phisher's server was returning the page upon receiving the HTTP headers, not an HTTP 100/Continue and waiting for the data. Curl 7.12.2 seems to send the data anyway, but later versions don't (though if the file is small enough, curl seems to send it).

So it seems to be a bug at the phisher's web site (now down).

One other thing, though ... sending the data to another spammer's form, which DOES return an HTTP 100/Continue header - curl sends the HTTP headers, gets the HTTP/100 header and sends the data. However, the "-v" option to show data sent and received by curl does NOT show the data sent. It is found in a trace-ascii dump, however.

----------

So my bad. The phisher's server was broken (not asking for the data and depending on the size, curl may or may not send it anyway). The "-v" option does not show data sent in a POST (well it does in the case of curl 7.12.2, apparently, which seems to send the data even without the HTTP/100 header). I should have caught the missing "Continue" and tried another location to post the data.

Sorry to have bothered you. Curl is quite a useful programme.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2005-04-10 17:14

Message:
Logged In: NO

What happens (this is using http instead of https):

Curl makes a POST connection:

POST /aspnet_client/system_web/1_1_4322/_fOnlineBanking_fDefault.aspx.php HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)
Pragma: no-cache
Accept: */*
Host: onlinebanking.bankofoklaoma.com
Content-Length: 1024
Content-Type: application/x-www-form-urlencoded
Expect: 100-continue

(this length, 1024, is after truncating the DAT1 file to 1024 bytes)

but does not include the data

If I use the original DAT1 file, the Content-Length is listed at 5218 (correct) but no data is sent.

If I truncate the file to 1023 bytes, the Content-Length is set to 1023 and the data is posted.

(that is from checking curl's output on stderr with the -v option as well as checking with tcpdump/ethereal)

So, for some reason, it looks like the limit is 1K.

Is it a limit of my connection?
Loading a local copy of the page and using the hosts file (the original hostname does not resolve - it was a phisher site that is now down) to set the IP address - the page with the form which sent the data, Mozilla sends with a Content-Length of 5207 (well, I changed the fake username and password I used to test the site) in a POST and sends the data (as checked via tcpdump).

Let me check a command line client, w3m, to use the local page and send the data ... again, it sent the data in the POST for the larger files.

Let me use ethereal to extract exactly the POST sent by w3m and use that as the DAT file in curl.

While w3m made the POST connection and sent the data, curl makes the POST connection and does not send the data in the large file.

I have tried version (7.13.1, patched for the problem with trying to read the entire enrtopy file when the file has not end, such as /dev/urandom) and that has the same problem. I will try some earlier versions until I find one that works and post here.
I am on a dialup with an MTU/MRU of 576 (but that should not matter)

So ... I have mozilla and command line clients (at least w3m) s

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2005-04-10 16:54

Message:
Logged In: YES
user_id=1110

"fails" ?

How? What does curl say? What does the server say?

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2005-04-10 16:47

Message:
Logged In: NO

Truncating the DAT1 file to 1023 bytes gets the shortened data to POST.
Truncating the DAT1 file to 1024 bytes fails.

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2005-04-10 15:08

Message:
Logged In: YES
user_id=1110

curl can still POST just about any amount of data.

What happens when you try this? What is the limit would you
say? Can you provide a way we can try that repeats the problem?

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=1180358&group_id=976
_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-tracker
Received on 2005-04-11