[scribus-dev] Change to UTF-32

[Andreas Vox]

> Am 30.05.2016 um 22:10 schrieb Craig Bradney <cbradney at scribus.info>:
> 
> 
>> On 30 May 2016, at 22:08, Khaled Hosny <khaledhosny at eglug.org> wrote:
>> 
>> On Mon, May 30, 2016 at 09:39:11PM +0200, Craig Bradney wrote:
>>> 
>>>> 
>>> 
>>> What about speed? We already need to speed up the text engine massively, including rendering (or especially rendering).
>> 
>> Converting a QString read from 25MB text file to UTF-32 takes <
>> 0.4 seconds, so that does not seem to be a concern. The current slowness
>> is mainly bad handling of text spanning multiple frames more than
>> anything else and Andreas is working on a fix for this.
>> 
>> 
> 
> 
> http://unicode.org/faq/utf_bom.html#UTF32 has some fairly strong wording about slow down from using UTF32.
> 

I wonder what they’d say if they saw our code in ScText_Shared…

As long as we pack every Char in an ScText struct, 8 vs. 16 vs. 32 bit doesn’t matter. A better structure would use a rope data structure or a list of strings for text and store style information as runs outside the text (coming soon).

My main issue is that we won’t be using QString as a model for story text. String has many handy member functions that QLsit<uint> doesn’t. It’s also possible that QList<uint> is slower for many tasks.

But if everyone is fine with the change, we can make it work. Text search and nextCharacter() have to be rewritten anyway in order to handle non-latin text correctly.

The file format will probably use UTF8 anyway. Importing/exporting text will always handle character encoding as needed.

/Andreas