UFRaw: Are the Highlights in Your Raw File Really Blown?

I love the convenience and speed of UFRaw. But sometimes UFRaw creates areas of blown highlights in an image, when the raw file itself doesn't have any blown pixels. The only way to avoid or mitigate these blown highlights is to process the image with the wrong white balance or use a different raw processor. I have verified this problem in UFRaw versions 0.16 through 0.20.

Written February 2010. Updated (checking the latest version of UFRaw) March 2015.

An example of falsely blown highlights

The image below on the left is the red channel of flower.cr2 (available for download if you want to follow the steps on this page) after raw processing with UFRaw. The image below on the right is the red channel after raw processing with dcraw. Note the complete lack of detail in the UFRaw red channel in the brightest areas of the flower petals (circled in red). In stark contrast to the UFRaw output, the dcraw red channel shows full detail in the red channel highlights.

Below is a screenshot showing the UFRaw settings used to process flower.cr2:

Please do note that the raw file was shot using UniWB white balance. This is not the correct white balance for processing the raw file. The image was shot in full daylight and so the correct white balance is "daylight", which multiplies the red channel by 2.301 (UniWB multiplies all channels by 1.0). If you don't select the "daylight" white balance, you will see the wrong colors and you won't see any problem with falsely clipped highlights.

From the UFRaw histogram in the screenshot above, it appears that the red channel is blown. However, Rawnalyze shows that raw file flower.cr2 has at most a few blown pixels in the green channel and no blown pixels in the red or blue channel:

Additional observations about UFRaw highlight handling

The number of blown pixels depends on both the input and the output color profile.

As you can see by the screenshot below, different combinations of input and output profiles in UFRaw give differently-shaped histograms with a spike or peak in the highlights:

Some profile combinations — in particular combinations using sRGB as the output space — have an abrupt, steep "peak" of pixels rather than mapping all the "blown" pixels to the same solid shade of gray. This peak of pixels, which are all mapped to nearly instead of exactly the same shade of gray, is still a loss of information, compared to dcraw's output which maintains full differentiation in the shades of gray in each channel. It strikes me as weird that UFRaw's sRGB output does a better job of preserving the red channel details than the UFRaw ProPhotoRGB output, given that ProPhotoRGB has a much larger color gamut than sRGB.

Highlight recovery produces peaky histograms

When activating the UFRaw highlight recovery (which should not be necessary for a raw file with no blown pixels), the region of affected pixels gets mapped to a small, peaky hill instead of being mapped according to actual pixel values.

No pixels are blown in flower.cr2 and so highlight recovery should not be needed. However, using UFRaw soft or sharp highlight recovery on flower.cr2 maps the region of the otherwise "blown" (artificially blown by UFRaw, not blown in the raw file itself) pixels into a small, peaky hill. Again, information is lost because the pixels in the small, peaky hill are wrongly mapped to nearly the same shade in the red channel, instead of being mapped according to their actual values.

Changing the color balance multipliers changes the proportion of blown pixels.

Moving the UFRaw temperature slider up or down, which effectively raises/lowers the red channel multipliers while lowering/raising the blue channel multipliers, drastically affects the number of blown pixels. For this image, if the temperature is lowered sufficiently, no pixels are blown in any channel. For some images, as one channel ceases to have blown pixels, another channel starts to have blown pixels.

As you can see from the above screenshot, raising the white balance color temperature slider makes the pixel spike higher (more pixels are blown). And lowering the temperature slider to 4217 (which has the same effect as lowering the red channel multiplier and raising the blue channel multiplier) eliminates the blown pixels.

This "disappearance of the blown pixels" when lowering the temperature slider was my first clue that something was wrong with how UFRaw is handling image highlights. Changing the white balance multipliers has no effect on whether the pixels in the raw file are actually blown or not: you should always be able to use negative exposure compensation to counteract the effect of applying higher multipliers to the channels of a raw file. But in UFRaw, negative exposure compensation just moves the histogram spike farther to the left.

Exposure compensation, sensor saturation

The codebox below shows the dcraw command line that was used to process flower.cr2:

See where the dcraw command line says "-S 6500"? The "-S" option in dcraw allows you to override the dcraw default "full well" or sensor saturation value, which is 3726 for the Canon 400D (from whence came flower.cr2).

The dcraw "-S" option in effect allows you to apply negative (use a value higher than the default) or positive (use a value lower than the default) raw exposure compensation. It should be noted that if you do have clipped channels in your raw file and you use a full well value higher than the default (which varies between camera models), the output image will have false colors in the highlights. So raw negative exposure compensation needs to be used with caution. But in an image like flower.cr2, raw negative exposure compensation avoids artificially clipping the red channel because of the color balance multipliers (there are other dcraw command line options that can be used to prevent artificial channel clipping, but this isn't an article about dcraw).

How to modify UFRaw to eliminate falsely blown highlights

It is easy to mitigate (but not eliminate) the UFRaw peculiar treatment of highlights if you are willing to compile a special version of UFRaw from source. Open the UFRaw source code file "dcraw.cc" and find the entry in the Adobe camera coefficients table that correspond to your camera model.

For the Canon 400D, the entry reads as follows:

The modified entry reads:

"e8e" is hexadecimal for 3726, which is the sensor saturation (full well) value for the Canon 400D. Change whatever your sensor saturation value is to the equivalent hexadecimal value for 4095 (if your camera has a 12-bit A-D converter) or 16383 (if your camera has a 14-bit A-D converter). Just so you don't have to look it up for yourself, hexadecimal for 4095 is fff, and for 16383 is 3fff. If you'd like to do your own conversion, EasyCalculation has very nice on-line converters.

After making the above change, compile and run UFRaw as usual. The images below show that the modified UFRaw now holds detail in the flower highlights, and the histogram is smooth.

Most likely you wouldn't want to use your modified UFRaw on images that really do have blown highlights.

As you can see, UFRaw, when compiled after making the suggested change to the Adobe camera coefficient table in dcraw.cc now keeps all the details in the highlights. (I also eliminated the UFRaw Canon "normalization" code — see note 4 below — but that modification is not relevant to this article.)