In Quest of Well Behaved Working Spaces

What might prevent the standard RGB working spaces from being well behaved?

It's somewhat disheartening to find that so many widely distributed RGB working space profiles aren't completely neutral, or at least are not neutral to the 6 decimal places that the Argyllcms xicclu utility displays. So I wrote a c code program that uses LittleCMS version 2.5 ("LMCS2") to make working space profiles and started experimenting to see what might make any given working space profile well behaved or not well behaved.

All the standard RGB working space profiles are matrix profiles. There are only a few bits of crucial information that fully determine an RGB matrix profile. These are:

1. The profile's tone response curves ("tone reproduction curves", "TRCs")
2. The profile's red, blue and green unadapted primaries
3. The profile's source white point
4. The profile's header illuminant

In theory, if a profile maker gets all of the above information correct while making an RGB working space profile, one would hope that the resulting profile is well behaved. In practice, it turns out that when making an ICC profile, quantization from decimal-to-hexadecimal conversions affects the resulting ICC profile.

It's not the tone response curves or the primaries.

A well behaved RGB working space profile's tone response curves ("TRCs") should be equal in all three channels, start at zero and end at max white. All of the proprietary and open source profiles that I included in my survey of 30 standardly available working space profiles meet these simple TRC conditions.

Since the TRCs are not the reason why the standardly available working space profiles are not well behaved, the next thing I looked into was the profile primaries. It turns out that as far as documentation on the world wide web goes, there is pretty much universal agreement on the primaries for all the standard working RGB color spaces. The only exception is the WideGamutRGB profile, for which Pascale and Lindbloom give slightly different values. Profiles made using Pascale's WideGamut primaries match older WideGamut profiles distributed by Adobe and Canon, so I will guess that Pascale's primaries are correct.

Checking to see whether any actual ICC profile uses the wrong set of unadapted primaries means using matrix math to "reverse chromatic adapt" the profile's adapted primaries from the profile's D50 illuminant back to the original source primaries via the profile's source white point values in the profile's white point tag (for V2 profiles made according to V2 specifications) or chromatic adaptation matrix chad (for profiles made according to V4 specifications). I set up a spreadsheet do just that. But the goal of this article is to figure out whether it's even possible to make well-behaved profiles using the correct unadapted primaries. So the rest of this article uses published values for the unadapted profile primaries.

The white points seemed to be a large part of the problem (but weren't)

A plethora of white points

On the one hand, everyone pretty much agrees on the right TRCs and unadapted primaries to use when making the various RGB working space profiles. One the other hand, there is considerable official and unofficial disagreement as to the correct D50 and D65 white point values:

Table 1 notes:

1. Bruce Lindbloom, Chromatic Adaptation"; According to Lindbloom, the C, D50, D65, and E values are from ASTM E308-01. I used Lindbloom's XYZ to xyY equations to produce the corresponding xy values.
2. Bruce Lindbloom, T to xy.
3. Wikipedia, White points of standard illuminants.
4. Calculated from XYZ values given in the ICC V4 profile specifications, using Lindbloom's XYZ to xyY equations.
5. The profile header illuminant X, Y, and Z values were determined using IccXml and converted to xyY values using using Lindbloom's XYZ to xyY equations.
6. Reference Input/Output Medium Metric RGB Color Encodings (RIMM/ROMM RGB), paper for presentation at PICS 2000 Conference, March 26-29, 2000, Portland, Oregon.
7. From selected well-behaved old V2 D50 and D65 profiles. The white point tag X, Y, and Z values were determined using IccXml and converted to xyY values using using Lindbloom's XYZ to xyY equations.
8. When used with the sRGB unadapted primaries, these white point values make a well behaved color space profile. But it's not the real sRGB color space profile because these white point values are not the values given in the sRGB specifications.
9. From the lcms version 2.5 API documentation for making an sRGB profile.
10. Adobe® RGB (1998) Color Image Encoding.
11. See Wikipedia's article on sRGB and also the references given at the bottom of the Wikipedia article. In particular, see the 1998 IEC 61966-2-1 fourth working draft and the 1996 W3's A Standard Default Color Space for the Internet - sRGB. The two references give different white point values for sRGB.
12. Compare the values in the rows highlighted in yellow with the values in the immediately preceeding rows. These values have been quantized by hexadecimal rounding during the profile making process.

The official ASTM E308-01 white points versus the Correlated Color Temperature white points

Whew! Table 1 shows a plethora of official and not-so-official white point values to choose from! So I picked two sets of values that seemed reasonably "official", depending on who you consult: Lindbloom's ASTM E308-01 values, and the Correlated Color Temperature values, which I calculated using Lindbloom's "T to xy" equations. Then I made two sets of test profiles. Here are the results:

Looking at Table 2 above, it turns out that when using Lindbloom's ASTM white point values, the Apple and ColorMatch profiles and also Lindbloom's WideGamut profile are well behaved. And when using the Correlated Color Temperature white point values, the Apple, ColorMatch, Identity, and PAL profiles are well behaved. That leaves the Adobe, ProPhoto, sRGB, and WideGamut as not well behaved when made using using either white point.

What about all those other white point values?

Using only Lindbloom's Chromatic Adaptation white point values for all the profiles didn't result in a full set of well behaved profiles. Using on the Correlated Color Temperature white point values didn't result in a full set of well behaved profiles. The temptation is to keep trying all possible white points and variations thereof until you find just the right white point that makes a well-behaved profile.

The trouble with the "trying all the white points" approach is that the standard RGB working spaces are (supposed to be and usually are) defined by published specifications. These profile specifications tell you not only what unadapted primaries to use when making the profile, but also the precise source white point values to use.

Unfortunately, not every standard working space has specifications that are readily available on the World Wide Web. For some profiles (Apple, ColorMatch, WideGamut) formal specifications don't seem to be available at all anymore, if they ever were. For some profiles (Adobe, ROMM/ProPhoto, sRGB, etc), specifications are easy to find on the internet. For the video profiles (PAL and such) the complete specifications can be obtained through official standards organizations. Finally, although the Identity profile is a mathematically obvious color space, it's not a color space that's ever been proposed as an image editing color space, and so there aren't any official specifications.

Table 3 shows the result of making RGB working space profiles using the white point that seems like the correct white point based on easily available specifications and/or existing well-behaved older V2 profiles:

Quantization from hexadecimal conversion turned out to be the real problem

Having run out of other candidates for the source of the remaining deviation from being well balanced as shown in Table 3 above, that leaves hexadecimal rounding as the culprit. Examining Table 3, one value leaps off the page (because I highlighted said value in yellow). Except for ColorMatch (which deviate in both the a* and b* channel), all the profiles that aren't yet well behaved to six decimal places have exactly the same remaining deviation, which is b*=0.001233 instead of b*=0.000000.

A hexadecimal rounding eror?

ICC profiles don't actually use decimal numbers to encode the profile illuminant, primaries, and white point information. Rather they use hexadecimal numbers. There are unavoidable quantization errors in the conversion of decimal numbers to hexadecimal numbers and back.

The recurring value "0.001233" in Table 3 above happens to be equal to a deviation in the equivalent XYZ "Z" value (that is, deviation from the Z value for a completely well balanced profile) of exactly 0.000015. If you multiply 0.000015 by 65535, the result is 0.983025, which oddly enough to those of us who don't speak hexadecimal, equals 0 in hexadecimal. And if you multiply 0.000016 by 65535, the result is 1.000000, which equals 1 in hexadecimal. So the decimal value "0.001233" does look like a hexadecimal rounding error.

Actual profile illuminant values vs the ICC D50 specifications

Given that profile illuminant values are determined by the ICC Profile Specifications, it's not too surprising that the ICC profiles in this little study (the open source and proprietary profiles that I examined and also the profiles I created using lcms) all have exactly the same profile illuminant. But it is surprising (unless you already know about hexadecimal quantization) to find that the profile illuminant values revealed by iccToXml are not equal to the illuminant D50 X, Y, and Z values that were used to create the profiles in the first place:

Given the discrepancies between the official ICC D50 values and the actual illuminant values found in actual ICC profiles, that looks like a "decimal to hexadecimal to decimal" rounding error. And sure enough, it is. Starting with the ICC D50 specification XYZ values, here's the round-trip conversion through hexadecimal and back:

starting values:         (0.964200000, 1.0, 0.824900000)
convert to hex:          (F6D4, FFFF, D32B)
convert back to decimal: (0.964187076, 1.0, 0.824887465)
absolute difference:     (0.000012924, 0.0, 0.000012535)

According to iccToXml, for all the ICC profiles that I examined, the actual profile header illuminant XYZ values are (0.964202881, 1, 0.824905396). Here's the round-trip conversion through hex:

 starting values:         (0.964202881, 1.0, 0.824905396)
 convert to hex:          (F6D5, FFFF, D32C)
 convert back to decimal: (0.824902724, 1.0, 0.964202335)
 absolute difference:     (0.000000546, 0.0, 0.000002672)

Notice that the illuminant values found in actual ICC profiles convert to hex and back with a smaller error than the D50 values given in the ICC specifications, and also that the actual profile illuminant hexadecimal values are larger by 1 hexadecimal unit (F6D4->F6D5, D32B->D32C). So it seemed logical to try modifying the LCMS2 sourcecode by replacing the ICC D50 specification XYZ values with the illuminant XYZ values found in actual ICC profiles.

Modifying the LCMS2 source code to use the actual profile illuminant values

Here's how to modify the LCMS2 source code to use the illuminant values found in actual ICC profiles:

1. Download the LCMS2 source code from sourceforge (or git if you are adventuresome).
2. Locate and open the "lcms2.h" file in the downloaded source code "include" folder.
3. Locate the lines that read:

   // D50 XYZ normalized to Y=1.0
   #define cmsD50X             0.9642
   #define cmsD50Y             1.0
   #define cmsD50Z             0.8249

4. Modify these lines to read:

   // D50 XYZ normalized to Y=1.0
   #define cmsD50X               0.96420288
   #define cmsD50Y               1.0
   #define cmsD50Z               0.82490540

5. Compile LCMS2 as usual and install it as root.
6. Recompile your ICC profile-making c code.
7. Make a new set of working space profiles using the white points that are specific to each profile as indicated in Table 3 above.

After making this small modification to the LCMS2 source code, and then recompiling my profile making code with the white points listed in Table 3, here's the result:

After modifying the LCMS2 source code to use the illuminant values found in actual ICC profiles rather than in the ICC profile specifications, the resulting Adobe and PAL profiles are now well behaved to six decimal places. And except for ColorMatch, all the D50 profiles are still well behaved. But the sRGB didn't change. And the Apple and ColorMatch profiles are less well-behaved than they were before modifying the LCMS2 source code.

Unfortunately hexadecimal rounding affects not just the profile illuminant, but also the adapted primaries and the profile white point (or chad, for V4 profiles and V2 profiles made according to V4 specifications). Some combinations of "source white point plus unadapted primaries" are more susceptible to hexadecimal rounding than others. To make well behaved sRGB, Apple, and ColorMatch profiles requires addressing all of the values affected by hexadecimal rounding, not just the profile illuminant.

As of version 2.5, LCMS2 doesn't have any provision for compensating for hexadecimal rounding when making ICC profiles. Profile-making code in Argyllcms version 1.6.3 does compensate for hexadecimal rounding. It's possible to make a profile using Argyllcms and then use the resulting adapted primaries to calculate "pre-quantized" unadapted primaries that when used with LCMS2, will exactly reproduce the Argyllcms adapted primaries, thus making well-behaved ICC profiles that fit available profile specifications.

When using pre-quantized primaries, there's no need to modify the LCMS2 source code to use the D50 illuminant values found in actual ICC profiles. However, as an important aside, a benefit of modifying the LCMS source D50 XYZ values to match the values found in actual profile illuminant values is that for relative colorimetric conversions to the LCMS2 built-in CIELAB color space, LCMS2's "transicc" will output values that match Argyllcms' "xicclu".

The white point isn't the problem after all

After discovering that quantization errors play a major role in making RGB working space profiles not completely well-behaved, I experimented with using Argyllcms 1.6.3 to generate RGB working space profiles using a variety of primaries and white points. So far I haven't been able to produce a profile that isn't well behaved to six decimal places. So the only time the white point affects whether the resulting profile is well behaved or not, is when quantization errors are not accounted for during the profile making process.

Conclusions

And thus ends my investigation as to why so many RGB working space profiles aren't well behaved:

• Many widely distributed RGB working space profiles are made using the wrong source white point and possibly even the wrong unadapted primaries, which might accidentally produce a well behaved profile, or might not. But the white point values per se are not the problem. Rather the problem is hexadecimal rounding: some combinations of "source white point plus unadapted primaries" are more susceptible to hexadecimal rounding errors than others.
• If a profile maker starts with the right source white point and primaries, LCMS2 version 2.5 can be used to make many (not all) D50 RGB working space profiles that are well behaved to six decimal places. But D65 profiles won't be well behaved because of hexadecimal quantization errors. As an aside, C and E profiles also won't be well behaved unless quantization errors are accounted for during the profile making process.
• As of version 1.6.3, Argyllcms makes profiles with adapted primaries that have been compensated for quantization caused by hexadecimal rounding. The Argyllcms adapted primaries can be used to calculate "pre-quantized" unadapted primaries that can be used with LCMS2 to make well-behaved RGB working space profiles.

If you make or use floss ("free/libre and open source") ICC profiles, you might be interested in my Survey of Free and Open Source ICC RGB Working Space Profiles. Also see Use LCMS to Make Your Own ICC Profiles, where I provide the LCMS2 code and some pre-quantized primaries for making yourself a set of RGB working space profiles that are (1)well behaved and (2)made using the correct source white point.