cURL / Mailing Lists / curl-users / Single Mail

curl-users

Re: Curl --data-urlencode posts broken non-English characters

From: Dan Fandrich <dan_at_coneharvesters.com>
Date: Tue, 2 Feb 2016 21:48:04 +0100

On Mon, Feb 01, 2016 at 11:18:37PM +0800, Irvin Jacob wrote:
> I am using curl to reply to a topic in a vBulletin forum from a text file.
>
> My command looks like this:
>
> curl -b "cookies.txt" -d "title=" --data-urlencode "message_backup_at_message.txt"
> --data-urlencode "message_at_message.txt" -d "wysiwyg=0" -d "iconid=0" -d "s=" ...
> "http://somesite.com/newreply.php?do=postreply&t=something" -L
>
> Note that I just wrote "..." above because the command is too long.
>
> message.txt has Japanese text "大日本帝國" inside and it is encoded in UTF-8.
> Curl successfully posted the message but the Japanese characters turned into "
> 大日本å¸åœ‹" in my message posted on the forum. Other normal (ASCII)
> characters look fine though. I also tried other foreign characters like Greek
> letters and accented letters. "Thére Àre sôme spëcial charâcters ïn thìs têxt"
> became "Thére Àre sôme spëcial charâcters ïn thìs têxt"
>
> I tried specifying the header Content-Type: application/x-www-form-urlencoded;
> charset=UTF-8 via -H option. I tried adding --compressed and --tr-encoding
> options. I also tried different versions of curl. All of these did not change
> anything.
>
> Could somebody please help me?

It's quite possible that the server doesn't support UTF-8, but assumes the
input is always ISO-8859-1 or something. Are you sure the server can even do
what you want it to? Does it work if you use a browser? If so, then you can
see what headers and data the browser sends in this case and make curl match.

>>> Dan
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2016-02-02