Some of my Chinese characters change to different unknown characters when posting story in html mode after upgrading to GL1.3.10. It doesn't happen in staticpages. I am trying to find out if there is any new features that make story to reformat the content when posting?

I have been testing all night and couldn't find the clue. Has anyone upgraded and tested with Japanese or some other Asian utf-8 characters?

Sam

Got an example link?

Hi Turias,

It happens to all 1.3.10 sites, even the English utf-8 ones, if I post Chinese on them.

Here is one example.

I posted "ÇëÉԵȣ¡" and the last character changed to something unknown.

Oops this forum does not accept utf-8 characters.

Sam

So it's only some of the characters? Strange. Are you sure those characters are available in UTF-8 and not only in a specific Chinese charset?

Yes, I am sure they are utf characters because that's how my windows xp is setup, with multi-lingual utf-8 pack.

And when I posted the same characters to 1.3.9sr1 sites, there was no problem, and even on the staticpages on 1.3.10 sites, there was no problem.

Sam

OK, I reinstall 1.3.9sr2 on the previous example site, and the Chinese character comes up with the same problem.

So it is not the 1.3.10's problem. I am trying to narrow down the problem, but so far still haven't got any clue.

Sam

It's possible that the kses filter is messing with things here. At least we have a bug report to that effect (it's for Japanese, though).

bye, Dirk

I remember the kses problem, but it was too long a go, and resolved. Plus, the current problem is not sitewide because the language works very well with the StaticPages. The problem is only with story.php or related.

Now, I have narrowed down to this situation:

I installed a fresh 1.3.9.sr1 on the site and it works well with the Chinese characters. So it is some changes between sr1 and sr2 that generated this issue.

Euan told me that his Japanese site is working fine and he is using PHP5. Mine is PHP4.3.9. Could that be a problem?

I am going to test story.php one by one from CVS, to see at which point it started this problem.

Sam

I tested almost every file that I think as anything to do with story, but couldn't pin point the problem.

The only sure thing is somewhere between 1.3.9sr1 and 1.3.9sr2 that the problem started.

Sam

Looking at the (very short) list of changes for 1.3.9sr2, the only thing that comes to mind is that I had to edit the language files to remove the $topic variable.

Is it possible that the language files are somehow damaged / messed up?

bye, Dirk

That's a good thought. Let me check it out.

Sam

Nope! I keep forgetting the logic that StaticPages are working fine. If it is the language file the StaticPages should be effected.

I generated an English_utf-8 file out of the default English langauage file. That should be the cleanest utf-8 file, and it still mess us some utf-8 chinese characters. But, on the 1.3.9sr1 sites using English_utf-8, you can type in Chinese utf-8 characters with no problem.

The other Chinese Big5 and GB codes language files work. But then they are not unicodes.

The problem is isolated to story "body" files in html only. Even the story title works, and plain old text format works too. So the problem is it changes some utf-8 characters to others on HTML format. Does it have to do with html clean up functions?

Sam

It is kses!