[Scribus] Bounty for true PDF handling in Scribus

[Craig Ringer]

Hi

You probably meant to write this URL:
    http://www.cofundos.org/project.php?id=94

I'm glad to see someone interested in this, and looking at the 
(corrected) URL, it looks like you're putting enough cash behind it to 
interest people in working on it.

I have a few tips for anybody who might be interested in tackling this 
project; see below.

>       * Import of PDF file: Selection which page is imported
>   
Some of the support required for this is already in place; in 
particular, LoadPDF(...) in util_ghostscript.cpp can already accept page 
numbers, and the Navigator class is capable of passing PDF page numbers. 
Last time I looked at this, most of the work required was in 
customfdialog.{cpp,h} .
>       * Scaling: The imported PDF page can be scaled
>       * Cropping: The imported PDF page can be cropped
>   
You will want to embed the page as a PDF XObject when outputting PDF 
from Scribus. This handles some interesting issues quite neatly:

    - Graphical elements supported by PDF and not by Scribus will be 
handled correctly

    - There's no need to parse the content stream and convert it to 
Scribus objects

    - You don't have to extract fonts from the PDF and convert them to 
local fonts (this is often not even possible as they're frequently
      embedded as subsets)

    - You don't have to convert raster graphics (which might be in 
unusual formats or use unusual compression, like CITT Fax encodings)
       to something Scribus can understand and then recompress them 
(often losing quality) during export.

    - .... and lots more.

There are however some issues with the XObject approach:
   
    - If the document being imported is of a PDF version newer than the 
PDF being exported, problems may arise.
    It's probably best to rasterise the input PDF (like we do now) in 
this situation. In theory you could scan the
    content streams of the input PDF and check if it actually used 
features not supported by the current PDF
    version (like transparency and layers if you're exporting PDF 1.3) 
but this is difficult and may not be entirely
    reliable.

    - Using XObjects only helps if you're exporting to PDF. You'd want 
to use GhostScript or similar to convert
    the PDF to EPS for embedding in PostScript output (this will 
preserve vector lines etc), and you'd need to
    rasterise it when exporting to raster formats like TIFF.

    - You'll still need a low-res raster image of the PDF for a 
placeholder. The current code does a fine job there, so
    this probably won't be much hassle.

    - I'm not sure how to handle non-PostScript mixed vector/raster 
output formats (like GDI printing).
    Jean Ghali may have some suggestions about this area.

    - Scribus's export settings will not apply to the data from the 
imported PDF, so if (eg) the
    input PDF uses RGB colours and the user is exporting CMYK from 
Scribus, you'll land up with
    RGB colours in the final PDF from the XObject. It is hard to detect 
or fix this without actually
    scanning the PDF content streams and resources and rewriting them 
(which is a dramatically harder
    job than just inserting an XObject). This is probably the biggest 
roadblock, and a good fix for it
    would require code that scanned the input PDF and did things like 
re-compressed images, checked
    font embedding, and converted colours based on the export specs. 
However, except for embedding
    a rasterised copy of the PDF as we do now, I don't  see much of an 
alternative.

    A PDF with mixed RGB and CMYK colour is quite valid and OK, as is 
one with duplicate copies
    of fonts, etc. The only issue here is that if the user must satisfy 
a particular spec, they must separately
    ensure that their input PDFs also satisfy that spec.

To embed PDF pages as XObjects in a Scribus PDF, you'd need to parse the 
PDF document (using a third party library), extract the content 
stream(s) attached to the desired /Page object, and recursively scan the 
resource dictionaries of the content streams to find the required 
objects. When writing the page's contents to the Scribus PDF you'd 
assign new object numbers to all the resources (since the original 
numbering might conflict with numbers Scribus has used) and update the 
indirect references, then you'd write all the objects out to the Scribus 
PDF. When that's done you can write an XObject (see the PDF Reference) 
that draws the content streams extracted from the original document, 
applying any scaling and other transforms that you require.

Teh bulk of the the work involved would be in pdflib.cpp, though you'd 
probably have to work on ScImage as well.

Scribus already uses the PoDoFo PDF library for importing Illustrator 
documents, so it might be an option for your PDF parsing requirements. 
PoDoFo can be used to extract the objects and their dependencies, 
renumber the objects and generate byte-sequence representations of them 
that can be written out to the Scribus PDF document stream. You'd then 
just need to add the required objects to Scribus's XRef tables and 
insert code into the page's content stream to draw the XObject.

There is a tool shipped with podofo, podofoimpose, that uses this 
XObject method to do simple PDF imposition. It is likely to serve as a 
reasonable example of how you can do things, though since it uses podofo 
to write out the resulting PDF it's far from an exact match to the 
Scribus PDF embedding task. If someone working on this does use podofo 
I'd be happy to help answer questions, look into any bugs or issues 
encountered, etc.
>       * Exporting of Scribus file as PDF: The imported PDF page's colors
>         (RGB and CMYK), vector elements, texts including their font,
>         links etc. are preserved
>   
This will be handled if you use XObject embedding, though take note of 
the caveats about mismatched versions, colour spaces, etc mentioned above.
>       * User friendly GUI: Handling of the imported PDF should be
>         tightly integrated in the existing user interface
>   
This mostly involves alterations to the file dialog preview panel (to 
enable page selection for PDFs) and the properties palette.
> I know from discussions in IRC that these features are more or less
> planned for the next release. However, we plan to layout our next
> booklet in May 2008. Therefore we need the feature to be implemented
> within the official SVN repository by April 30, 2008.
>
> What do you think, it this feasible?
Probably not, but I'd be overjoyed to be surprised. Some of the code 
involved, particularly pdflib,
is difficult to work on and might require more changes than you'd 
expect. However, the task its self
actually isn't *that* difficult if you're happy to accept the XObject 
limitations outlined earlier, so it's
far from impossible.
> Are the Scribus developers willing to help us?
I don't really count as a Scribus dev anymore (I don't do much these 
days) but I'd be happy to offer
some help to anybody working on this. I've done a bunch of work on 
podofo, so if it gets used I
might be of some use helping out with that side of things.

--
Craig Ringer