[scribus-dev] Change to UTF-32

[Khaled Hosny]

On Mon, May 30, 2016 at 08:56:06PM +0200, Craig Bradney wrote:
> 
> > On 30 May 2016, at 17:26, Andreͮaͦsͯ <andreas.vox at gmail.com> wrote:
> > 
> > Hi!
> > 
> > Khaled recently committed a change to the HOST-Oman/ctl branch that changes the text coding from UTF_16 (QChar/QString) to UTF-32:
> > https://github.com/HOST-Oman/scribus/commit/b2886e64b4aba660e4939d6cdaba066aed39d081
> > 
> > Since we have divided opinions on this issue, would you mind reviewing the discussion in the comments and add your own thoughts?
> > 
> > Best regards
> > /Andreas
> > 
> > 
> 
> 
> I don’t really see the difference because I don’t know the code. If
> the APIs enable me, or anyone who doesn’t know the text code to do
> stuff like insert text at a certain point for example, the it would
> seem ok.
> One thing that is mentioned is for example inserting a QChar.. so how
> does one programmatically insert 1 character?

StoryText has no API to insert single character, inserting QChar
apparently works because it can be cast to QString or some such. You can
now either insert a QString and the code will convert it to UTF-32, or a
QVector<uint> of UTF-32 code points. We might have a friendly typedef
for this vector.

> I don’t have an opinion on the 16 vs 32 as long as it works so we can
> do all operations.. read/write to file, display it, type it, export it
> to various formats.

The big difference is that UTF-16 is variable length, a Unicode
character can be represented by one or two UTF-16 code units, but
Scribus was treating it as a fixed length encoding which is wrong.
UTF-32, on the other hand, is fixed length encoding, so the assumption
all over Scribus codebase can be kept.

Regards,
Khaled