[scribus] Colour Profiles - make colour matching worse!

[Hal V. Engel]

On Monday 27 July 2009 04:33:06 am John Beardmore wrote:
> Hal V. Engel wrote:
> > On Sunday 26 July 2009 04:45:11 pm John Beardmore wrote:
> >> Peter Nermander wrote:
> >>> Also RGB and CMYK are totally different. When you tell Scribus to
> >>> produce a PDF for print, it creates a CMYK PDF. But from what you are
> >>> writing (that your printer is using its built in profile) I think your
> >>> printer wants RGB data, so you should create a PDF intended for screen
> >>> (the only difference is RGB vs CMYK).
>
> I think this terminology causes confusion because most desktop printers
> don't need CYMK ?

Almost all printers are at the base level CMY(K) devices.   The reason that 
you think these devices are RGB is because many of the drivers hide the 
inherent nature of the device by accepting RGB input that is converted into 
CMYK by the driver.  This RGB to CMYK conversion can also be handled by the 
CMS if the driver will accept CMYK input (like the GutenPrint divers do) and 
on my Linux box I do this with my printers. This is also how this is handled 
by most professional printers in their RIP software (IE. they use a CMS to 
create device specific CMYK separations). 

>
> I would have though some words along the lines of
>
>    * RGB for general purpose PDF output, typically for web use
>      and desktop printing.
>
>    * CYMK PDF, typically for commercial printing.
>
> >>> This page doesn't cover everything, but is has links to further
> >>> reading. http://en.wikipedia.org/wiki/Color_management
> >>>
> >>> Color management is VERY complicated, and you have to make sure you
> >>> understand which profiles are needed for which translation. I know
> >>> enough about color management to understand that I shouldn't be using
> >>> it yet... I need to learn more.
> >>
> >> Same here, but I suppose the frustration is that the documentation of
> >> most software doesn't make colour handling particularly clear.
> >>
> >> It would seem logical that if CIELAB is a colour space that can describe
> >> anything the human eye can perceive, that all images be stored
> >> internally in this format. This then requires a profile that describes
> >> the conversion of each input devices colour space to CIELAB, and another
> >> profile for each output space describing the conversion from CIELAB to
> >> the printer specific output.
> >
> > This is what the color management engine does although there is no
> > requirement that images or anything for that matter be stored in CIELab
> > or any other device independent color space.
>
> Which is possibly a large part of the problem as the variety of possible
> profiles and incompatibilities must go up rapidly with the number of
> possible input and output devices in the absence of a common internal
> format.

The use of a common internal STORAGE format does not change the number of 
profiles required in any way.  All devices need to be characterized no matter 
what internal format is used to store images.  In addition users can use the 
existing CMS facilities to store everything in CIELab if they want to.  This 
has issues outside of the CMS however - for example image editors may not 
handle this very well.  If there are not profiles available for each device 
then there is no way to convert from the device's native color space (there 
are NO devices that natively produce CIELab images so you need a CMS to do 
this) to the "common internal format".  In addition there are no 
incompatibilities in the existing CMS design since the CMS does in fact use a 
common internal format called the Profile Conversion Space (PCS) when doing 
conversions between device color spaces.  This has been a feature of the ICC 
specification since the very beginning.  

>
> >  Profiles can also use CIEXYZ as the Profile
> > Conversion Space (PCS) as well as CIELab and the CMS knows how to handle
> > conversions between CIELab and CIEXYZ so that it can handle conversions
> > that involve either or both PCS types.
>
> OK -  if there are only two PCS types, that's a start.

And that start happened with the very first version of the ICC specification 
and has not changed.

>
> >> For PDFs which are frequently multi use things, it would presumably be
> >> sensible to keep images in the CIELAB format then use profiles for each
> >> VDU, CRT, LCD and mobile phone used to view them, or each windows or
> >> press printer that might be used to put them on paper.
> >
> > In fact the PDF specification calls for support for various color spaces
> > including what the spec calls CIE Based color spaces.  One of these is
> > CIELab but this also includes objects where the color space is specified
> > by an ICC profile.  In addition the spec allows for what it calls CalRGB,
> > CalCMYK and CalGray color spaces which are basically specified by a
> > combination of an XYZ white point and some gamma curves (actually a gamma
> > curve formulas) and the XYZ values of the primaries.  Internally when a
> > PDF is sent to a device for display/printing the PDF spec calls for
> > everything not in the device color space to be converted from what ever
> > color space the individual objects are in to CIEXYZ and then from CIEXYZ
> > to the output device color space.   It does not really make sense  to
> > store everything in the PDF in CIELab since the specification calls for
> > everything to be converted to CIEXYZ as an intermediate step and as long
> > as all of the objects in the PDF have well specified color spaces then
> > the conversion process is unambiguous.  As a side note anything in a
> > "device" color space is in fact not well defined in most cases and this
> > is were many issues pop up.
> >
> >> In the absence of such clear and well defined data flow,
> >
> > As you can see from the above the color data flow is well defined and
> > very clear but is also complex.  But I think what you were trying to say
> > is that this is not clear to users.
>
> Yes -  to a significant extent.  Though I suspect that is CMS were to be
> redesigned from scratch, we might end up with something less complex as
> well as more accessible.
>
> Apart from the disk space used, I don't really see anything wrong with
>
>      profiles from devices to
>      universal colour space to
>      profiles to output devices

But this is exactly what we have now.  The only thing you want to change is to 
require that images be stored in the "universal colour space".  The whole 
point of the CMS is that storing images in a "universal colour space" is not 
necessary since all it does is shift where some of the color transformation 
processing is done.

That is the flow would go from (current flow):

stored image (input device color) --> PCS
PCS --> output device color

to (your flow):

input device color --> PCS (stored image)
PCS (stored image) --> output device color

CMS processing that results in a transformation either into or from the PCS is 
indicated by --> and to do these one profile is required.   Notice that each 
flow involves two transformations to get from input device color space (think 
camera or scanner) to output device color space (think monitor or printer) and 
the only difference between these is when the input device to PCS transform 
happens not if it happens.  There is no difference in complexity and it 
appears to me that your proposal is a solution in search of a problem that 
does not exist.

>
>
> The fact that "the PDF specification calls for support for various color
> spaces" may be a reflection of history rather than what's needed to get
> the job done.

To some extent this is true.  The Cal* color spaces are basically left over 
from the PostScript specification and I suspect would not exist if the PDF 
specification where being designed in a vacuum after the ICC specification 
where widely accepted.  I also find the "Device" color spaces to be worse than 
useless since the ambiguity that they create is actually harmful.  In effect 
by using the PDF "Device" color spaces users are turning off color management 
in their PDF files.  But this does not mean the the CIE Based part of the PDF 
specification is incorrect or overly complex because it is, in fact, based on 
the ICC specification and it is actually the most modern part of the color 
specification in the PDF format.   I would be happy with the PDF specification 
if it did not have the Cal* and Device elements since it would simply the 
format and possibly make it more accessible to users and it would make things 
simpler for programmers as well.  But the Cal* and Device color spaces predate 
the CIE Based color spaces and are included so that older PDF files continue 
to work. 

>
> Of course, I don't expect the historical legacy to go away, but I do
> wonder if the CMS world could converge on something better.
>
> >  Which leads to
> >
> >> the mumblings
> >> about the complexity of CMS seem to contribute to a small of bad design.
> >>
> >> Is CMS something that needs to be complex, or just something that's got
> >> complex because of the lack of standard methods and clear description /
> >> specification ?
> >
> > The CMS needs to be complex.  It is the nature of the beast.  The ICC
> > specifications are clear and are based on science dating from the 1920's
> > and 1930's dealing with human perception of color.  This by it's nature
> > is complex because it actually deals with how the human mind works.
>
> So why does anything more complex that the full gamut of what is visible
> to the eye need to be described ?  Surely the point of CMS is to
> reproduce original colours accurately, not to interpret them ?
>
> > It should be possible to hide at least some of this complexity from
> > users. The real issue is that at present users are forced to deal with at
> > least some of this complexity and app and OS vendors/developers have not
> > done a good job of shielding users from it.
>
> Or describing it.
>
> >  In fairness this is difficult because of the
> > complexity and because many subsystems are involved in the process.  OS/X
> > is probably the best at hiding this complexity but it still has issues in
> > this regard.  Windows is only slightly better than Linux and other unix
> > variants in this area and both are far behind OS/X with respect to user
> > accessibility. There is lots of work being done on the *nix side to help
> > with this issue (there are a number of GSoC projects currently underway
> > for example) but it will likely be a while before a significant amount of
> > this effort filters down to end users.
>
> I suppose if a full CMS requires huge complexity and an understanding of
> human perception, maybe there is scope to implement something which is
> much simpler and just reproduces the colours accurately.

But this is not possible.  Human perception is one of the main issues that 
this is designed to deal with.  To illustrate if you are in a building with 
incandescent lighting and a person in front of you has a white shirt on the 
shirt will appear white even though the lighting has a color temperature of 
around 2800K (IE. the light is yellowish but we still see a white shirt).  If 
you follow that person outside into the sun light the shirt will still appear 
to be white even though the light now has a color temperature of around 5000K 
and the physical color of the light reflected off of the shirt has changed 
significantly.  Why do I bring this up?  Because it is an easy to reproduce 
example of how two totally different physical colors (meaning the color of the 
light reflected off of a "white" object) can appear to be the same color 
because of human perception.   The human mind instantly compensates for this 
change in lighting and even though the actual physical color of the objects 
changes as the lighting changes our perception of these colors does not 
change.  The CMS is designed with this and other aspects of human perception 
in mind and there is no way to avoid this.  It is just the underlaying nature 
of what we are dealing with and it is not possible to ignore it.

>
> >> In some ways it would be great to get into CMS and have everything come
> >> out as it should all the time, but for many, the pragmatic approach is
> >> to tweak curves a bit in gimp taking account of your experience with the
> >> target printer.
> >
> > Well actually the correct approach is to understand and correctly use the
> > CMS.
>
> In most businesses the correct approach is to convey the information in
> a document of which the pictures may be a small part. Brownie points are
> awarded for doing fast and doing it pretty.
>
> Except in very specialist businesses, management does not award points
> for the use of CMS, nor more formally correct use of CMS.
>
> > It is very powerful and if you understand how it works and how to
> > correctly set things up it will have a very positive impact on the
> > quality and consistency of your work as well as actually streamlining
> > your work flow.
>
> Yes -  it's that streamlining that attracts me to it, and the notion
> that if I print to a different device I can get the same colours.
>
> > Having said that it takes a lot of up front effort to do this which makes
> > getting past the learning curve a difficult task for many users.
>
> Yes -  and the need to buy kit to calibrate monitors and print output
> doesn't help either.

You do need to base this on actual measurements of the hardware being used or 
it does not work.  Nature of the beast.  It was only a few years ago that the 
vendors of this hardware viewed this as a specialized activity that only those 
involved professionally in graphics and printing would use.   For example way 
back only Macs were used for this type of work and the hardware and software 
needed for profiling was only available for Macs.  It used to be that the 
hardware was very expensive (think $1,000 - inflation adjusted - for basic 
hardware to be able to calibrate and profile monitors and $5,000 for 
printers).  Now you can get the hardware to calibrate and profile monitors for 
about $60 and basic printer profiling hardware for about $400.  In addition 
many printer,  paper and ink vendors now give away high quality printer 
profiles on their web sites and if your OS/printer driver/printer/paper/inks 
are supported this way then proper printer profiles are a no cost option.  
Most windows and OS/X users should be able to find suitable printer profiles 
on-line for free.  For most *nix users this is not an option.  If you can't 
find suitable profiles on-line there are printer profiling services that will 
create custom profiles for as little as $25 (it wasn't that long ago that 
these services charged over $125).  Even though there are some costs here they 
are not out of the reach of most users and many users should be able to setup 
everything they need for under $125 (monitor measurement device + purchase a 
printer profile + profiling target for camera and/or scanner) to have a 
complete correct CM work flow.  And there are open source solutions for 
profiling software that work on Windows, OS/X and all of the open source 
platforms like Linux.

>
>
> Cheers, J/.