skip to main content

Models for image editing: Display-referred and scene-referred

This article uses the xyY reference color space to explain the similarities and differences between display-referred and scene-referred image editing. Even though the two models serve very different image editing goals, both models work with bounded RGB data. Display-referred RGB data is bounded by Color, which is to say by both Luminance and Chromaticity. Scene-referred RGB data is bounded only by Chromaticity.

Written May 2014. Updated August 2014.

Introduction: Display-referred and scene-referred image editing both use bounded RGB data

Display-referred editing and scene-referred editing are two different models for editing images. Even though the two models serve very different image editing goals, both models work with bounded RGB data. Display-referred RGB data is bounded by Color, which is to say by both Luminance and Chromaticity; scene-referred RGB data is bounded only by Chromaticity.

This article uses the xyY reference color space to explain the similarities and differences between display-referred and scene-referred image editing.

Separating Luminance and chromaticity: XYZ and xyY

RGB ICC profile working color spaces are defined by the locations of their primaries (red, green, and blue profile colorants) in the XYZ reference color space. In the XYZ reference color space, Y carries Luminance information, and X, Y, and Z together carry information about color.

In the present discussion, it's convenient to separate Luminance information from color information, which is what the xyY reference color space allows us to do. xyY is calculated from XYZ by the following equations:

  • x = X / (X+Y+Z)
  • y = Y / (X+Y+Z)
  • Y = Y

In the xyY reference color space:

  1. xy specifies color, more technically speaking "chromaticity", less technically speaking, whether a color is red, green, blue, etc, and also how saturated the color might be, but carries no information about how bright the color might be:
    • Less saturated colors have xy values that are closer to the D50 white point xy values.
    • More saturated colors have xy values that are farther from the D50 white point xy values.
  2. Y, also called "Luminance", specifies how intense (bright) a color might be.
    • The higher Y is, the brighter and more intense the color is.
    • The Luminance of a color never goes below zero because there is no such thing as a displayed color that's less bright than solid black (R=G=B=0.0). This is a mathematical reflection of the fact that out there in the real world, there's no such thing as "less than zero reflected light" (black holes out in space aren't relevant to the present discussion).

ProPhotoRGB and sRGB in the xyY reference color space

Figure 1 shows the ProPhotoRGB and sRGB color spaces displayed on the same chromaticity diagram:

The sRGB and ProPhotoRGB chromaticity xy values plotted on the xy chromaticity diagram:
  • The xyY reference color space is three-dimensional, just like the XYZ reference color space is three-dimensional. The xy chromaticity diagram's "point of view" is looking straight down the Y axis. So the Y axis sticks up from the intersection of the x and y axes at (0.0, 0.0).
  • The sRGB primaries (colorants) chromaticity (xy) values are:
    1. Red xy (chromaticity): x=0.65, y=0.33.
    2. Green xy (chromaticity): x=0.32, y=0.60.
    3. Blue xy (chromaticity): x=0.16, y=0.07.

    The lines connecting the sRGB chromaticities form a triangle that traces the boundaries of the sRGB color gamut's allowable xy values. Put differently, sRGB colors are bounded by the triangle that's created by the sRGB chromaticities.

    Any xy values that are outside the sRGB triangle are out of gamut with respect to the sRGB color space. These colors are "redder than sRGB red", "greener than sRGB green", and so on.

  • The ProPhotoRGB primaries (colorants) chromaticity (xy) values are:
    1. Red xy (chromaticity): x=0.73, y=0.27.
    2. Green xy (chromaticity): x=0.16, y=0.84.
    3. Blue xy (chromaticity): x=0.04, y=0.00.

    The lines connecting the ProPhotoRGB chromaticities form a triangle that traces the boundaries of the ProPhotoRGB color gamut's allowable xy values. Put differently, ProPhotoRGB colors are bounded by the triangle that's created by the ProPhotoRGB chromaticities.

    Any xy values that are outside the ProPhotoRGB triangle are out of gamut with respect to the ProPhotoRGB color space. These colors are "redder than ProPhotoRGB red", "greener than ProPhotoRGB green", and so on.

  • For both the sRGB and ProPhotoRGB color spaces, and also for all other RGB ICC profile working spaces, the xy values for black (the black dot in the center of the chromaticity diagram) and white (the white dot in the middle of the black dot in the center of the chromaticity diagram) are:
    1. Black xy (chromaticity): x=0.35, y=0.36.
    2. White xy (chromaticity): x=0.35, y=0.36.

As already stated, the farther the xy coordinates of an RGB color are from the D50 white xy coordinates, the more saturated the color is. Comparing the sRGB "reddest red" xy coordinates to the ProPhotoRGB "reddest red" xy coordinates, ProPhotoRGB's reddest red is redder, that is, more saturated than sRGB's reddest red. The same is true of ProPhotoRGB's greenest green and bluest blue, compared to sRGB's greenest green and bluest blue.

As an aside, ProPhotoRGB's greenest green and bluest blue are so saturated that they are completely outside the realm of real colors; this was done deliberately so that the ProPhotoRGB color space would be large enough to accomodate all possible print- and film-related colors. As another very important aside, ProPhotoRGB predates today's digital cameras and is not large enough to accomodate all possible colors produced by interpolating camera raw files and applying an appropriate camera input profile.

Luminance (Y)

This is the same chromaticity diagram as in Figure 1 above, except I tilted the xy plane so I could draw in the Y axis and also put in the Y values for the sRGB and ProPhotoRGB primaries (the lines are not to scale!).

Notice that the Green Y values are about the same for both color spaces (0.71 vs 0.72). But the ProPhotoRGB color space's Red Y value is a little higher (0.29 vs 0.22). And the Blue Y value is 0.00 (actually 0.00007629), vs the sRGB Blue Y value of 0.06.

In an ICC profile color managed workflow, the correct way to calculate the Y value (Luminance) of a linear gamma RGB color (Luminance can't be calculated using perceptually uniform RGB values) is to:

  1. Multiply the color's red channel value by the color space's D50-adapted red chromaticity's Y value, which for sRGB is 0.22.
  2. Multiply the color's green channel value by the color space's D50-adapted green channel chromaticity's Y value, which for sRGB is 0.72.
  3. Multiply the color's blue channel value by the color space's D50-adapted blue channel chromaticity's Y value, which for sRGB is 0.06.
  4. Sum the resulting three values.

So, for example, in the sRGB color space, the Luminance of solid white=(1.00, 1.00, 1.00), is (1.00 times 0.22) plus (1.00 times 0.72) plus (1.00 times 0.06), which equals 1.00. In other words, the Y value of solid white is 1.00.

Also in the sRGB color space, the Luminance of solid black=(0.00, 0.00, 0.00) is (0.00 times 0.22) plus (0.00 times 0.72) plus (0.00 times 0.06), which equals 0.00. In other words, the Y value of solid black is 0.00. Nice how that works out, yes?

I'll give two more examples for calculating Luminance in the sRGB color space:

  • The Luminance of sRGB's greenest green=(0.00, 1.00, 0.00) is (0.00 times 0.22) plus (1.00 times 0.72) plus (0.00 times 0.06) which equals 0.72.
  • The Luminance of sRGB's reddest red=(1.00, 0.00, 0.00) is Y=0.22, which should be obvious from looking at the formula for calculating Luminance.

I won't write out the formulas for you, but it should be obvious that in the ProPhotoRGB color space:

  • The Luminance of solid white is Y=1.00.
  • The Luminance of solid black is Y=0.00.
  • The Luminance of the reddest possible ProPhotoRGB red=(1.00, 0.00, 0.00) is Y=0.29. So the reddest possible ProPhotoRGB red has just a tiny bit higher Luminance value than the reddest possible sRGB red, which has a Luminance of Y=0.22.

If you think about it for a second, it should be obvious that in every RGB working space, the Luminance of solid white is Y=1.00, and the Luminance of solid black is Y=0.00.

Comparing the sRGB and ProPhotoRGB working color spaces in terms of their xyY values:

  • In terms of Luminance (Y)
    1. The Luminance of ProPhotoRGB's reddest red (Y=0.29) is slightly higher than the Luminance of sRGB's reddest red (Y=0.22). Speaking in less technical term, ProPhotoRGB reddest red is just a tiny bit brighter than sRGB red. Speaking more technically, ProPhotoRGB reddest red is slightly more intense than sRGB reddest red.
    2. The Luminance/intensity of ProPhotoRGB's greenest green (Y=0.71) is almost the same as the Luminance of sRGB's greenest green (Y=0.72).
    3. The Luminance/intensity of ProPhotoRGB's bluest blue (Y=0.00007629) is much less than the Luminance of sRGB's bluest blue (Y=0.06).
  • In terms of saturation (xy)
    1. Because its xy coordinates are farther from the D50 xy coordinates, ProPhotoRGB's reddest red is redder, that is, more saturated, than sRGB's reddest red.
    2. Because its xy coordinates are farther from the D50 xy coordinates, ProPhotoRGB's greenest green is greener, that is, more saturated, than sRGB's greenest green.
    3. Because its xy coordinates are farther from the D50 xy coordinates, ProPhotoRGB's bluest blue is bluer, that is, more saturated, than sRGB's bluest blue.
  • In terms of boundedness, both the ProPhotoRGB and the sRGB working spaces are bounded:
    1. sRGB colors are bounded by the triangle of xy coordinates that's created by the sRGB chromaticities.
    2. ProPhotoRGB colors are bounded by the triangle of xy coordinates that's created by the ProPhotoRGB chromaticities.

Display-referred editing is bounded image editing in a user-chosen RGB working space

RGB values below are given as floating point values. To calculate the corresponding integer values, multiply by 255 or 65535 as appropriate and round to the nearest whole number.

What is display-referred image editing?

The phrase "display referred" comes from the fact that ICC profile color managed image editing revolves around displaying images on devices. The displaying device might be a monitor, or an image printed on paper, or some other display technology. Regardless of the technology, when you display an image on a device, that device has a maximum and minimum brightness. The maximum and minimum brightnesses are the display-referred colors "white" and "black".

Correspondingly, when editing an image in a normal ICC profile color managed workflow using an RGB working space such as sRGB or ProPhotoRGB, "white" means the RGB color (1.0, 1.0, 1.0). This color has the very special significance that there's no such thing as "brighter than white". So in display-referred image editing, all RGB channel values are less than or equal to 1.0 and no color is brighter than "solid white", (1.0, 1.0, 1.0).

Similarly, "black" means the RGB color (0.0, 0.0, 0.0). This color has the very special significance that there's no such thing as "less bright than black". So in display-referred image editing, all RGB channel values are greater than or equal to 0.0 and no color is less bright than "solid black", (0.0, 0.0, 0.0).

When working with 8- and 16-bit integer display-referred RGB data, editing operations are coded to clamp RGB channel data to keep channel values within the display-referred range between 0.0 and 1.0 (multiplied by 255 or 65535 as required to produce integer values). For example, if you add solid white to solid white, you get what you started with, which is solid white: (1.0, 1.0, 1.0) plus (1.0, 1.0, 1.0) = (2.0, 2.0, 2.0), which is immediately clamped to (1.0, 1.0, 1.0). And if you add solid red to solid red, you get solid red: (1.0, 0.0, 0.0) plus (1.0, 0.0, 0.0) = (2.0, 0.0, 0.0), which is immediately clamped to (1.0, 0.0, 0.0).

As will be shown below in terms of the xyY reference color space, the colors that can be represented in display referred image editing are always bounded by the color gamut of the RGB working space in which the values are encoded.

Display-referred RGB image editing and the resulting xyY values

This is the same chromaticity diagram as in Figure 2 above, except I removed the ProPhotoRGB chromaticities and Y values, and I added the chromaticities and Y values for the brightest and most saturated possible sRGB magenta, cyan, and yellow (the lines are not to scale!).

The Luminances (Y values) of the brightest and most saturated possible sRGB magenta, cyan, and yellow are calculated as follows:

  • The Luminance of yellow (1.00, 1.00, 0.00) is (1.00 times 0.22) plus (1.00 times 0.72) plus (0.00 times 0.06), which equals 0.93.
  • The Luminance of magenta (1.00, 0.00, 1.00) is (1.00 times 0.22) plus (0.00 times 0.72) plus (1.00 times 0.06), which equals 0.28.
  • The Luminance of cyan (0.00, 0.00, 1.00) is (0.00 times 0.22) plus (1.00 times 0.72) plus (1.00 times 0.06), which equals 0.78.

Now imagine that you drew on the tilted chromaticity diagram more vertical lines at the appropriate xy locations for all possible sRGB colors (for 8-bit images, that's 256 times 256 times 256 equals 16777216 different vertical lines, so you wouldn't really want to do this). Imagine that you made each vertical line the height of the corresponding Y value. So, for example, the ArgyllCMS xicclu utility would tell you that:

  • The 8-bit sRGB color (10, 200, 85) has the xy values (x=0.29, y=0.46) and the Y value (Y=0.59).
  • The 8-bit sRGB color (77, 128, 235) has the xy values (x=0.28, y=0.29) and the Y value (Y=0.48).

If you keep drawing in new vertical lines over the xy values corresponding to various combinations of R, G, and B in the sRGB working space, the tops of all the vertical lines will begin to form a sort of "Y canopy" that graphs the Luminance (Y values) of all possible sRGB colors. And you can do the same thing for the ProPhotoRGB working space and any other RGB working space.

Fortunately, you don't need to do this to see the "Y canopy" for different RGB working spaces because Bruce Lindbloom has already done it for you. Scroll down to his 3D Gamut Viewer and choose:

  1. sRGB as the Primary Working Space
  2. "None" as the Secondary Working Space
  3. xyY as the Color Space

Rotate the resulting 3-D representations to view the "Y canopy" (not an official color management term!) from all angles. Note that the xy values never stray outside the triangle traced on the xy plane by the color space's reddest red, greenest green, and bluest blue chromaticities (profile colorant xy values). This is easiest to confirm by viewing from the sides, with the xy plane perpendicular to the screen. From the top, with the Y axis pointing toward the screen, perspective makes it look like the top of the canopy extends past the xy triangle, but it really doesn't.

Now repeat again with the ProPhotoRGB color space.

Summarizing color-managed, display-referred RGB image editing

  1. No editing operation performed on clamped and bounded RGB data can create RGB values that are out of gamut with respect to the image's ICC profile color space. Expressed in terms of RGB values, editing operations that produce RGB channel values that are less than 0.0 or greater than 1.0 are simply clipped to 0.0 or to 1.0 in the affected channel(s). Expressed in terms of the xyY reference color space:
    • No editing operation produces a color that's so dark that its Y value is less than 0.0.
    • No editing operation produces a color that's so bright that its Y values extend past the "canopy" of Y values defined by calculating the Luminance of every possible combination of bounded RGB values. In other words, display-referred RGB data is bounded by the working space's "Y canopy".
    • No editing operation produces a color that's saturated enough to be located past the edges of the xy triangle created by the xy coordinates (chromaticities) of the RGB working space's colorants. In other words, display-referred RGB data is bounded by the working space's "in gamut" xy values.
  2. In theory, any RGB working space can be used to edit clamped and bounded integer RGB data. However, in practice:
    • For 8-bit integer clamped and bounded RGB data, the RGB color space needs to be perceptually uniform and small enough to avoid posterization. The regular sRGB is a good choice. With care, AdobeRGB can be used. Using linear gamma versions of any RGB color space will cause serious posterization in the shadows of an image, which means you can't do radiometrically correct image editing in an 8-bit integer color space.
    • For 16-bit integer clamped and bounded RGB data, even the very large gamut linear gamma ProPhotoRGB color space can be used for editing without fear of posterization, which means 16-bit integer clamped and bounded RGB data does allow for radiometrically correct image editing. Except, of course, if you perform an operation that would otherwise send the RGB data outside the bounded gamut of the color space (eg adding solid white to solid white).

Scene-referred editing is also bounded image editing in a user-chosen RGB working space

Why scene referred image editing?

With display-referred RGB data you have roughly two and half stops of head room above middle gray and maybe six and a half useable stops below middle gray, at which point the RGB data is too dark to visually discern the difference between solid black and "just barely gray". So at best you have 9 stops of dynamic range, compared to the 20 or more stops of dynamic range you might find in some (certainly not all!) real world scenes.

The usual solution to the dynamic range limitations of display-referred, hence clamped and bounded, RGB data is to allow the RGB channel values to be however high or low as is needed to encode the scene data. When using the OpenEXR file format's 32-bit floating point file format, as many as 30 stops of dynamic range are available, which is roughly equivalent to allowing an image's RGB values to go from a low of R=G=B=0.001 (just barely discernible from solid black), all the way up to R=G=B=1,000,000.

When working with color-managed scene-referred high dynamic range RGB data, the chromaticities from any RGB working space can be used. But the tone reproduction curve does need to be linear. Manipulating nonlinear RGB data produces radiometrically incorrect results, breaking the scene-referred nature of the data.

Editing scene-referred high dynamic range RGB data of course requires that there isn't any clamping code in the RGB editing operations. This means you can add (1.00, 1.00, 1.00) and (1.00, 1.00, 1.00) and get (2.00, 2.00, 2.00) or twice as bright as the maximum brightness allowed in display-referred editing.

And you can add what in clamped RGB editing would be referred to as solid red to what in clamped RGB editing would be referred to as solid red and get (2.0, 0.0, 0.0), which is twice as bright as the display-referred maximally saturated and bright red (1.0, 0.0, 0.0).

xyY values that result from unclamped editing operations with scene-referred RGB data

When working with scene-referred high dynamic range RGB data:

  1. Resulting Y values can easily be greater than 1.00. In fact 1.00 ceases to have any significance.
  2. Increasing the Luminosity (intensity, Y value) of an RGB color doesn't change the color's xy values:

    So for example, in the sRGB color space, if you multiply the reddest possible sRGB red by 7.0, the xy values don't change (the red doesn't get any redder!), but the Intensity, or Y value, is multiplied by 7.0, going from Y=0.22 to Y=1.56.

    And in the ProPhotoRGB color space, if you multiply the reddest possible ProPhotoRGB red by 7.0, the xy values don't change (the red doesn't get any redder!), but the Intensity, or Y value, is multiplied by 7.0, going from Y=0.29 to Y=2.02.

    In other words, in unclamped image editing, no matter how high an image's Y values might be, the image's xy values are still bounded by the color space's possible xy values as determined by drawing lines between the color space's chromaticities.

So, for example, ArgyllCMS's xicclu utility would show you that:

  • The floating sRGB green color (109.72, 800.25, 223.72) has the xy values (x=0.31, y=0.49) and the Y value (Y=611.67).But the xy values are still inside the sRGB xy triangle (see the white dot labelled "1" in the diagram to the right).
  • The floating ProPhotoRGB color (90.34, 1887.44, 223.72) has the xy values (x=0.18, y=0.73) and the Y value (Y=1369.65). But the xy values are still inside the ProPhotoRGB xy triangle (see the white dot labelled "2" in the diagram to the right).

Summarizing color managed scene-referred image editing

  1. The user can choose the RGB color space chromaticities in which to encode and edit the scene-linear RGB data. Posterization because of the size of the RGB working space isn't an issue with high bit depth floating point RGB data.
  2. Unlike display-referred image editors, scene-referred image editors are not programmed to automatically clamp the resulting RGB channel values to the range between 0.0 and 1.0, as such clamping code would destroy the scene-referred nature of the data. Nonetheless, the resulting RGB channel values always represent colors that are bounded by the range of xy values enclosed in the triangle of xy values formed by the user-chosen RGB working space's chromaticities.
  3. Scene-referred RGB data can contain, and editing operations can produce, RGB colors with corresponding Y values that greatly exceed the "canopy" of an RGB working space's "maximum channel value=1.0" Y values. For example, Y might be 2.00, 10.00, 1000.00, etc.
  4. The particular RGB color (1,0, 1.0, 1.0) still has the color "D50", but so do the RGB colors (2.0, 2.0, 2.0), (1000.0, 1000.0, 1000.0), and etc. So there is no such thing as "solid white". Correspondingly, the color (1.0, 1.0, 1.0) has no special significance beyond being on the RGB working color space's gray axis.
  5. "Solid black" still means "no light" and is still represented by the channel values (0.0, 0.0, 0.0). As an aside, it should be noted that some scene-referred image editors do allow user-initiated editing operations that produce RGB channel values that are less than 0.0 (for example subtract blue from 100% saturated yellow). In such cases it's up to the user to decide how to deal with any negative channel values that s/he might produce. It behooves the user to get herself out of this situation as quickly as possible because many editing operations on such data produce entirely meaningless results.

Conclusion: Display-referred and scene-referred — two models for image editing serving very different editing goals

Summarizing the differences and commonalities between display- and scene-referred image editing:

  • For both display- and scene-referred image editing, the minimum RGB channel value is 0.0, corresponding to "zero percent of the light" being emitted or reflected by a surface (hard to acheive in practice, but light traps come close; a light-tight box with no light source inside doesn't count because by definition there's no light inside the box to be reflected off of any surfaces that might be inside the box).
  • Display-referred image editing is oriented towards editing images for display on devices. Devices have a maximum brightness determined by the device itself. Correspondingly display-referred image editing has a maximum brightness represented by the color white, or R=G=B=1.0. The dynamic range of real world scene data can easily exceed the roughly 9 stops of useable dynamic range that are available in display-referred image editing. So for scene-referred image editing, in order to accomodate high dynamic range, scene-referred RGB image data, "white" has no special significance and there is no maximum RGB channel value.
  • In both display- and scene-referred image editing, the range of meaningful RGB values is bounded by the user-chosen RGB working space and specifically by the triangle of xy values created by drawing lines on the xy plane connecting the user-chosen working space's Reddest Red, Greenest Green, and Bluest Blue chromaticities.
  • For display-referred image editing the range of meaningful RGB values is further bounded by the "canopy" of allowable Y values for each of the enclosed xy values, with the maximum Y value being 1.00 for the color "white" where R=G=B=1.0.
  • For scene-referred image editing Y isn't bounded by the upper value 1.0, and hence the corresponding R, G, and B channel values are also not bounded by 1.0. "Solid white" becomes a meaningless concept and the color R=G=B=1.0 is merely another point on the neutral axis that extends from 0.0 theoretically to infinity. However, all Y values and the corresponding RGB channel values are always positive or zero, except in the case of user-initiated editing moves such as subtracting red from solid black or saturated blue from saturated yellow. In such user-initiated cases the user should know what s/he is doing and be prepared to deal with the resulting "colors". Adding and subtracting these "colors" is OK, but trying to multiply or divide them by a (non-neutral) color will produce meaningless results.