Floating Point Dcraw + DCB, AMaZE, LMMSE, AFD, & VCD
Part 2: How to use dcraw-float

If you've ever used dcraw, using dcraw-float should be pretty simple. The only thing is, dcraw-float outputs raw color, so you do need a camera profile.

Written in 2011.

If you are reading this page, I'm guessing you've already used dcraw at the command line. But a lot of the dcraw-float command line options are different from the default dcraw command line options.

You need a camera profile

dcraw-float is hard-coded to output raw color (the equivalent of using '-o 0' at the command line with default dcraw). So you need a camera profile. Where do you get one?

1. You can purchase an IT8 or other camera profiling target and use ArgyllCMS to make your own custom camera profile.

2. The UFRaw website makes available a number of Nikon and Canon camera profiles and has a list of other resources for obtaining camera profiles.
3. With a little bit of ingenuity you can use digiKam + wine + dcpTool + dcp2icc to extract a matrix ACR camera profile from a dng file. Then (unless someone has written a Linux icc profile editor — if so, please let me know!) you need wine + icc profile inspector + bless or another hex editor + an icc profile into which you copy your camera profile matrix values (or maybe Argyllcms to create a blank icc profile). I've done it. It's a lot of work. But it works.

   Update: Many thanks to Shalamander who has successfully ported dcpTool to Linux and is working on making a Debian package.

4. You can mathematically manipulate the camera inverse matrix in the adobe_coeff table in the dcraw source code to produce a camera matrix profile (if anyone is interested I can write up the procedure; fair warning: it involves matrix math).

It probably goes without saying, but you have to use an image editing program or a command line utility (lcms, imagemagick, and argyllcms all have them) to apply your camera input profile to dcraw-float's raw color output.

Setting the white balance multipliers, darkness, and saturation levels

Scaling and White Balance Options, Before Interpolation
-r  Set custom white balance multipliers, defaults to '-r 1 1 1 1'
-k   Set the darkness level
-S   Set the saturation level
-H [0,1]  Highlight mode: 0=clip for white highlights, 1=unclip (default) for maximum detail in highlights

Setting the white balance using white balance multipliers

The only dcraw-float white balance option is using '-r red green1 blue green2' at the command line. I use UFRaw to obtain white balance multipliers. UFRaw opens raw files up very quickly (unlike a lot of raw rendering programs) and has an excellent eye dropper for quickly setting a white balance. You can copy the UFRaw white balance multipliers from the UFRaw "options/configuration" dialog and paste them directly into your command line:

If you don't specify white balance multipliers, dcraw-float defaults to '-r 1 1 1 1', which will give you (for most, maybe all cameras) a greenish looking image unless you used an appropriate (real, not digital) color filter on your lens. Why greenish? Your camera sensor is more sensitive to green light than it is to red or blue light. The whole point of white balancing with the red, green1, blue, green2 multipliers is to mathematically "rebalance" the raw file's red, green1, blue, and green2 image array values so that white objects are rendered without an unwanted color cast.

Setting the darkness & saturation values

Darkness (e.g. '-k 256') works exactly like darkness in the default dcraw code. The actual code is a bit different, but the effect is the same. In case you are wondering, the darkness value doesn't set an image blackpoint. Rather it's how you tell dcraw that any sensel value less than the darkness level is actually sensor noise rather than image information. (It's very educational to play with setting darkness to 0. The image will look exceedingly strange because it needs to be re-color-balanced using the shadow end of your levels sliders.)

Saturation (e.g. '-S 3726') works a bit differently than it does in the unmodified dcraw code. The only effect using '-S #' has in dcraw-float is when you also use '-H 0', in which case the command line saturation value overrides the default sensor saturation value in the adobe_coeff table. For example, if your default sensor saturation value is 3726, using '-S 4095' means sensel values will be clipped to a maximum of 4095 instead of 3726. '-S #' values higher than your camera's default saturation value will probably produce non-neutral highlights. (Just in case you don't already know, "saturation" refers to sensor saturation — how much light enters a camera before your image has blown highlights — and doesn't have anything to do with how saturated the colors might be in the output image.

Deciding how to deal with the highlights

The command line highlight options works more or less the same as in the default dcraw code (the actual code is different). But there are only two choices, '-H 0' for clipped highlights and 'H 1' for unclipped highlights.

A lot of advice gets repeated so often in the various digital imaging forums that it takes on the aura of unassailable truth. One of those bits of advice is that you should "never" use 'H 1' because you'll have discolored highlights. On the one hand, it is absolutely true that IF you have blown highlights in your image, the "price" of using '-H 1' is that you'll probably (I don't know about all cameras) have discolored highlights.

On the other hand, using '-H 0' has a price, too: if your image does NOT have any blown pixels, using '-H 0' can easily cause the loss of highlight detail (and using '-H 1' with an image with no blown pixels does not cause discolored highlights). And what if your intention is to eventually make a black and white rendition of your image, in which case discolored highlights are of no particular concern? Using '-H 0' means risking the loss of highlight detail in the color channels that aren't saturated. Below is an example image where using '-H 0' means losing blue channel highlight detail:

I removed the "blend" ('-H 2') and "rebuild" ('-H 3+') options from dcraw-float. In the default dcraw code these options use the adobe_coeff table camera inverse matrix values and some complicated math. Recoding dcraw to use floating point in these code sections would have required more work than I was prepared to expend on command line options that I don't use very often, based on camera inverse matrix values that aren't derived from the camera matrix profile that I do use. Truthfully, I'll probably never get around to figuring out how to make the "blend" ('-H 2') and "rebuild" ('-H 3+') options work in floating point.

Interpolation options

Interpolation Options
-q Set the interpolation quality
  q=0 is default dcraw bilinear interpolation
  q=1 is default dcraw VNG interpolation
  q=2 is default dcraw PPG interpolation
  q=3 ought to be AHD, but really is LMMSE
  q=4 is Jacek Gozdz's DCB interpolation
    -Q   set number of DCB iterations (default=0)
    -E turns off the default DCB refinement
    -N 1 turns on FBDD noise reduction
    -N 2 turns on FBDD noise+chroma reduction
  q=5 is AMaZE demosaic algorithm
  q=6 is LSMME demosaicing algorithm - gamma_apply hard-coded as 0
  q=7 is AFD Adaptive Filtered Demosaicing interpolation
    -B  sets number of afd_passes (default=0)
  q=8 is VCD Color Demosaicing Using Variance of Color Differences
  q=9 is no interpolation (scale colors but don't interpolate image)
-f Interpolate RGGB as four colors
  (-f option always uses VNG interpolation, regardless of specified -q option)

The interpolation options are pretty self-explanatory. The only thing that might not be obvious is that Jacek Gozdz's FBDD ("Fake Before Demosaicing Denoising"), command line options '-N 1' and '-N 2' can be used with any algorithm, not just DCB. In my opinion, the results of using FBBD are pretty impressive (DCB is pretty impressive, too).

Output gamma options

Output Gamma Options
  -G  Set custom gamma curve (default = 2.20)
  -F Apply gamma curve using floating point (double) math (slow, possibly more accurate).
    The default gamma curve appplication uses a look-up table.
    For -G 1.0 (linear gamma) using or not using -F makes no difference.

The gamma you specify at the command line should be the gamma used in your camera profile, which is probably either 2.20 (the default dcraw-float gamma) or 1.00. Some camera profiles have a "gamma" with a slight toe-slope instead of a true power gamma. For most of these profiles, I'm guessing that you can safely ignore the toe-slope and adjust the shadow tones in post-processing.

dcraw-float gives you two options for applying the gamma curve: the look-up table is fast; using double floating point math is slower and slightly more accurate. The differences in the actual output are so small that even this self-confessed pixel peeper uses the faster option.

Other options

Other Options
-v        Print verbose messages
-e        Extract embedded thumbnail image
-i        Identify files without decoding them
-i -v     Identify files and show metadata
-z        Change file dates to camera timestamp

Print sample image values to screen:
   Using -v for verbose output automatically prints sample image values;
   -x,-y,-u allow you to choose which sample image points are printed to screen.
      -x   Sample row (default is row 0).
      -y  Where to start on the row,
         e.g. value of 0.25 starts one-quarter of the way across the row (default=0.0: beginning of row).
      -u  Starting from the first sample image point, how many values to print to screen (default: 20).

If you don't use '-v' at the command line, you still get a lot of information printed to screen, such as how many of your image values are either blown or fall below the noise floor, which section of code is currently being executed, and so forth (hopefully more entertaining and possibly more educational than watching the terminal box do nothing at all while your raw file is being processed).

If you use '-v' at the command line, in addition to the normal screen output, sample image values get printed to screen at various points as the image gets scaled, interpolated, and rescaled. You can control which sample image values are printed by using '-x', '-y', and '-u' at the command line. If you use '-q 9' (no interpolation) at the command line, along with '-k 0' (sets the noise floor to 0) and '-G 1.0' (uses linear gamma), you can eye-dropper the uninterpolated values in the output image and actually match some of the red and green pixels up to the last set of values printed to screen. Unfortunately, I haven't (yet, and maybe never will) figured out how to get the print-out to scan across the sensel rows in an intuitive fashion, picking up RGRG followed by GBGB.

Some comments on dcraw-float

dcraw-float only outputs 16-bit tiffs. Any default dcraw option not already covered, is either hard-coded (no user choice), or simply isn't available in dcraw-float.

The default dcraw code has an option, '-W', that disables the automatic scaling of your output image to set a white point of 65535 in a (16-bit image) over a small percentage of the brightest pixels in your image. The way the default dcraw code is written, not using '-W' at the command line means you WILL have blown pixels in the output image, even if the raw file didn't have any blown pixels.

The dcraw-float code takes a different approach. Before writing the image to disk, the image is automatically rescaled so that the maximum image value is 65535. In other words, if there is only one pixel in the image, say in the blue channel, that is brighter than all the rest of the pixels, that pixel's blue channel value is rescaled to 65535, and all the other pixel values are linearly rescaled proportionately. This rescaling is done at the very end of the image processing: post-white-balancing, post-interpolation, post-noise-reduction; the only remaining step after this final rescaling is to apply the gamma curve.

No pixels that weren't already blown in the raw file, are blown by this rescaling process. Rather this rescaling serves to get all the image values as far from 0 as possible while it is still a true 16-bit image, without clipping the highlights, making it easier to manipulate the shadow tones in post-processing.

All the options

Below is what you see when you type './dcraw-float' at the command line:

dcraw-float is a modified version of Dave Coffin's Raw photo decoder "dcraw" v9.07, dcoffin a cybercom o net
modified by Elle Stone, http://ninedegreesbelow.com, to use double floating point calculations

Scaling and White Balance Options, Before Interpolation
-r  Set custom white balance, defaults to '-r 1 1 1 1'
-k   Set the darkness level
-S   Set the saturation level
-H [0,1]  Highlight mode: 0=clip for white highlights, 1=unclip (default) for maximum detail in highlights

Interpolation Options
-q Set the interpolation quality
  q=0 is default dcraw bilinear interpolation
  q=1 is default dcraw VNG interpolation
  q=2 is default dcraw PPG interpolation
  q=3 ought to be AHD, but really is LMMSE
  q=4 is Jacek Gozdz's DCB interpolation
    -Q   set number of DCB iterations (default=0)
    -E turns off the default DCB refinement
    -N 1 turns on FBDD noise reduction
    -N 2 turns on FBDD noise+chroma reduction
  q=5 is AMaZE demosaic algorithm
  q=6 is LSMME demosaicing algorithm - gamma_apply hard-coded as 0
  q=7 is AFD Adaptive Filtered Demosaicing interpolation
    -B  sets number of afd_passes (default=0)
  q=8 is VCD Color Demosaicing Using Variance of Color Differences
  q=9 is no interpolation (scale colors but don't interpolate image)
-f Interpolate RGGB as four colors
  (-f option always uses VNG interpolation, regardless of specified -q option)

Output Gamma Options
  -G  Set custom gamma curve (default = 2.20)
  -F Apply gamma curve using floating point (double) math (slow, possibly more accurate).
    The default gamma curve appplication uses a look-up table.
    For -G 1.0 (linear gamma) using or not using -F makes no difference.

Other Options
-v        Print verbose messages
-e        Extract embedded thumbnail image
-i         Identify files without decoding them
-i -v     Identify files and show metadata
-z        Change file dates to camera timestamp

Print sample image values to screen:
   Using -v for verbose output automatically prints sample image values;
   -x,-y,-u allow you to choose which sample image points are printed to screen.
      -x   Sample row (default is row 0).
      -y  Where to start on the row,
         e.g. value of 0.25 starts one-quarter of the way across the row (default=0.0: beginning of row).
      -u  Starting from the first sample image point, how many values to print to screen (default: 20).