UTF-8, htmlentities, character encodings...

registered · Posted: Tue Apr 03, 2007 8:24 pm

I have a GCalender user who is entering text data in Vietnamese with a char set of UTF-8. Apparently the data makes it into the database table ok (he looked with phpMyAdmin). And once in the database, it displays OK. However, when he displays the form to approve the event in the admin area, GCalender reads the table row and outputs the text into a textarea, like this: <textarea> $text </textarea>. Before outputing the $text, however, I run htmlentities on it because I don't want any angle brackets inside $text to mess up the <textarea></textarea> tags.

He reports that once this form is displayed all the text is garbled up in the textarea.

Now I don't really know what UTF-8 is, lol, but I did a quick read on htmlentities, and I see it accepts a 3rd argument for the character set. I gave him a special version of the PHP code that puts in a 'UTF-8' string for that third argument. But no go, apparently...(I wish I could see this!!)

I am trying to get him to export his data and send it to me so I can take a look and try to reproduce the problem. But does this problem ring any bells with anyone?

When he says he is using charset of UTF-8 (unicode I presume) what does he mean? Is this a function of his browser, or has he added something to the generated HTML to tell a browser to use UTF-8? I see Firefox gives one the option to view pages in a variety of encodings, and also has an auto-detect option.

Thanks for any insights.

Sells PC To Pay For Divorce Joined: Posts: 5661

hello gremmy,
i know you like long lists and and reading... Laughing

a list - http://www.phpnuke-database.com/content/view/1028/88888906/

a story - http://vietunicode.sourceforge.net/main.html

Posted: Wed Apr 04, 2007 6:47 am

Thanks HW. The second link answered a couple of my questions, in particular the HTTP header and/or the META HTTP-EQUIV "Content-Type" tag in the HEAD tells browers what encoding to try and use.

I just wish this guy would get back to me with the actual data to try. Maybe I should just do a htmlspecialchars() instead of entities?

registered · Posted: Wed Apr 04, 2007 7:16 am

Quote:

Maybe I should just do a htmlspecialchars() instead of entities?

I have been considering that myself to be quite honest. I wish that I had time to search for it, but 64BitGuy had a long thread about this either here or over on his site (which I cannot find the right link for any longer. Sad

).

I just don't know what are, if any, the security ramifications.

Posted: Wed Apr 04, 2007 4:58 pm

I gave him a version that used htmlspecialchars() instead of htmlentities() and it worked for him. Confused

What are the security ramifications? As long as < and > get converted to entities what could someone do?

registered · Posted: Wed Apr 04, 2007 7:49 pm

I believe htmlentities covers a wider range of characters than htmlspecialchars. I don't expect there are security issues, all the majors ones that can be used are covered by htmlspecialchars. Maybe some obscure foreign language character set may require htmlentities

Posted: Wed Apr 04, 2007 7:58 pm

Like I said, 64bitguy seemed to think it was a better approach than using htmlentities(). That means alot in my book. Except, all of my web site addresses for him are coming up with "Account Disabled", so all that good knowledge and articles he had are potentially forever gone?

Posted: Thu Apr 05, 2007 7:54 am

Thanks for the feedback guys. I obviously would like to support as many character sets as I can, and it's easy enough to switch to htmlspecialchars()....I just wish I knew all the trade offs. It seems to be a reasonable thing to do if it works...I just wish I knew why one worked and the other didn't.