When profiling your camera with ArgyllCMS, what type of profile should you make?
Interpolated camera raw files, xicclu graphs, and profile channel curves are used to illustrate differences between five different types of camera input profiles: an Adobe Dng matrix profile plus four different types of ArgyllCMS profiles: a simple linear gamma matrix profile, a three-channel, 256 point shaper matrix profile; and two LookUp Table profiles.
Written March 2015. Updated June 2015.
An earlier version of this article was written in 2012 under the title
"Five Camera Input Profiles Applied and Compared".
Introduction: Comparing different types of ArgyllCMS camera input profiles
In this article,interpolated camera raw files, xicclu graphs, and profile channel curves are used to illustrate differences between five different camera input profiles: an Adobe Dng matrix profile plus four different types of ArgyllCMS profiles: a simple linear gamma matrix profile ("colprof -am"), a three-channel, 256 point shaper matrix profile ("colprof -as"); and two XYZ-PCS LookUp Table profiles ("colprof -ax").
The "take away" points from this article are as follows:
- For most of us, most of the time, a "general purpose" simple linear gamma matrix camera input profile ("colprof -am") made under full spectrum lighting (preferably lighting that is relatively close to D50), is the better choice when making a camera input profile. Linear gamma matrix camera input profile do assume that your camera sensor responds linearly to light.
- The other types of profiles that you can make with the ArgyllCMS colprof utility (-aG, -ag, -aS, -as, -ax, -al) will give you lower deltas (deviations from the values given in your target shot reference file), at the expense of your having made the assumption that your camera sensor doesn't respond linearly to light. So you really do need a good reason (for example, if you know that your camera doesn't respond linearly to light) to make and use one of these other kinds of camera input profiles. In particular, if you don't know all the ins and outs of making and using a LUT camera input profile (-ax, -al), don't make a LUT profile.
- Low deltas aren't the only thing to be concerned about when you make a camera input profile. You also should consider whether your profile actually represents the way your camera responds to light, or whether instead you are engaging in "curve fitting" by asking for lower deltas than are merited by your target chart. What a reasonable delta might be varies depending on a host of factors not touched upon in this article. But the profile xicclu curves can warn you if you've produced a grossly inappropriate camera input profile.
This article does presume that you shoot raw and you have some familiarity with using ArgyllCMS to make a camera input profile. If you are looking for information on making a good target chart shot, see How to Make a Camera Input Profile Target Shot. If you are looking for a step-by-step "how to make a camera input profile", see How to Make a Better Custom Camera Input Profile.
When making a custom camera input profile, don't forget to look at the xicclu graph
There are quite a few different ArgyllCMS tools for evaluating profile quality. One of those tools is xicclu:
xicclu can look up "individual color values forward or inverted though an ICC profile table . . . [and] plot the device value composition down the neutral axis [emphasis added]."
Profile tone reproduction curves and xicclu graphs
Among other things, a xicclu graph of an ICC profile shows you how much Red, Blue and Green you need at any point to get neutral gray in the CIELAB reference color space. In a well-behaved RGB working space, you get neutral gray whenever R=G=B (Figure 1 below, top row). For most device profiles such as (many, but not all) camera input profiles and monitor profiles (Figure 1 below, bottom row), if R=G=B, the resulting color is not neutral gray.
If you've never seen a xicclu graph before, the general shape of the curve depends on the general gamma of the color space profile. At one extreme, linear gamma color spaces have deeply curved xicclu graphs. At the other extreme, perceptually uniform color spaces have nearly straight-line xicclu graphs. Also, color balanced profiles have the Red, Green, and Blue xicclu curves right on top of each other, whereas profiles that aren't color balanced have Red, Green, and blue xicclu curves that are offset from one another.
Xicclu graphs for camera input profiles
Most camera sensors can be considered to be more or less linear in their response to light, except of course in the deep shadows where the signal-to-noise ratio is very low, and in the extreme highlights as the individual pixels approach the full well state.
See Procedures for Evaluating Digital Camera Sensor Noise, Dynamic Range, and Full Well Capacities; How to Take the Data for how to check your own system linearity, which means the camera sensor's response to light + in-camera signal processing + lens + shutter speed/aperture settings + whatever else might affect the linearity of the interpolated raw file's RGB values.
Most of us don't have the equipment that would be needed to test just the sensor reponse to light, and most of us don't really care. What we really care about is system linearity. System linearity means:
- Changing the aperture and shutter speed changes the actual exposure value in a predicable and linear fashion.
- The interpolated RGB raw color values (that is, the interpolated RGB values prior to application of a camera input profile and any subsequent digital editing) are proportional to the light entering the camera from the scene that we photographed.
So in this article, whenever I mention "sensor linearity" I really mean system linearity, and I'm specifically excluding cameras that are deliberately set up to capture high dynamic range scenes with just one exposure by modifying the signals that are recorded by the individual pixels.
When examining the xicclu graphs for the various types of camera input profiles that you might make using ArgyllCMS, ask yourself whether the graph represents reasonable behavior for your camera sensor. For most cameras, the xicclu graphs should be smooth and exactly or nearly linear. So if your camera profile xicclu graph channel curves criss-cross back and forth or deviate substantially from the xicclu graph for a camera input profile with linear gamma TRCs, you might want to reconsider your profiling parameters..
Also please note that reasonable xicclu curves for a camera input profile aren't the same as reasonable xicclu curves for a monitor, scanner, or printer-paper-ink profile. This article only discusses camera input profiles.
Five camera input profiles applied and compared
About the five camera input profiles compared in this article
Except for the Adobe Dng matrix profile, all the profiles compared below were made using ArgyllCMS. The ArgyllCMS profiles include a simple linear gamma matrix profile; a three-channel, 256 points per channel shaper matrix profile; and two LookUp Table profiles. The ArgyllCMS profiles were made using a 288-patch Wolfe Faust batch-measured C1 (for digital cameras) IT8 target chart.
The notes below each pair of images list the profiles being compared and also point out differences from one profile to the next. Even on a good monitor with a decent monitor profile, using a properly color-managed browser, the differences are subtle, so you might want to try dimming the room lights and zooming in with your browser for a better view. Better yet, download this Gimp XCF file, which has the sample images, as interpreted by the different input profiles, as individual layers. When you can compare the profile variations by hiding and unhiding each layer in turn, the differences are much easier to see.
As an aside, in addition to the four types of ArgyllCMS camera input profiles discussed on this page, ArgyllCMS can also be used to make single channel shaper matrix profiles, single and three channel gamma matrix profiles, and LookUp Table profiles that use the CIELAB profile connection space. Also, there are all kinds of target shot preprocessing you can do, before using ArgyllCMS to make a camera input profile, that will result in profiles that automatically add saturation and apply tone curve alterations that mimic various proprietary camera styles. None of the profiles compared on this page used any "pretty out of the box" target shot preprocessing.
Adobe Dng vs ArgyllCMS simple linear gamma matrix profile
Comparing the profiles as applied to an interpolated camera raw file:
Comparing the Adobe Dng and ArgyllCMS simple linear gamma matrix profile xicclu graphs:
ArgyllCMS simple linear gamma matrix profile vs ArgyllCMS three-channel shaper matrix profile
Comparing the profiles as applied to an interpolated camera raw file:
Comparing the ArgyllCMS simple linear gamma matrix profile and ArgyllCMS three-channel shaper matrix profile xicclu graphs:
ArgyllCMS three-channel shaper matrix profile vs well made ArgyllCMS LookUp Table profile
Comparing the profiles as applied to an interpolated camera raw file:
The ArgyllCMS three-channel shaper matrix profile compared to the well made ArgyllCMS LookUp Table profile xicclu graphs:
Well made vs poorly made ArgyllCMS LookUp Table profile
Comparing the two LUT profiles as applied to an interpolated camera raw file: Shadows
As you can see by comparing the two images just below, the poorly made LUT profile not only produces oddly miscolored shadow areas; but also ridiculously miscolored highlights.
Comparing the two LUT profiles as applied to an interpolated camera raw file: Highlights
Comparing the well made and poorly made ArgyllCMS LookUp Table profile xicclu graphs:
It's also easy to make ridiculously curve-fitting three-channel shaper matrix profiles:
What type of input profile should you make for your camera and why you probably don't want to make a LookUp Table profile
How many color patches do you need to make a LookUp Table ("LUT") camera input profile?
With respect to making a camera input profile, the ArgyllCMS documentation says:
The -am option will create a matrix profile with linear (i.e. gamma = 1.0) curves. This may be useful in creating a profile for a device that is known to have a perfectly linear response, such as a camera in RAW mode.
With regard to making LUT (LookUp Table) profiles, the colprof documentation says:
A cLUT base table profile using a PCS of XYZ can be created if -ax is used, and this may have the advantage of better accuracy for additive type devices (displays, scanners, cameras etc.), may avoid clipping for displays with a colorant chromaticity that can't be encoded in L*a*b* PCS space, and may give a more accurate white point for input devices by avoiding clipping of values above the white point that can occur in L*a*b* based cLUT input profiles. A disadvantage of this type of profile is that it can be a lot less robust if given a test patch set that is sparse, or too evenly spaced [emphasis added; I don't know what is meant by "too evenly spaced" as you do want your color patches to more or less evenly cover the color gamut of the scenes you want to photograph].
So the question is "How many color patches are needed for making a camera input matrix profile vs making a camera input LUT profile"? The ArgyllCMS documentation doesn't specify how many color patches you need to make a good LUT camera input profile, but it does give a range for the required number of color patches for making monitor and printer profiles. For monitor profiles, for which you might make a gamma (-ag) or matrix (-as) profile or a LUT profile, depending on how you want to use the monitor profile:
At a minimum, a few hundred values are needed. A matrix/shaper type of profile can get by with fewer test values, while a LUT based profile will give better results if more test values are used. A typical number might be 200-600 or so values, while 1000-2000 is not an unreasonable number for a high quality characterization of a display.
For printer profiles, for which LUT profiles are commonly made:
At a minimum, for an "RGB" device, a few hundred values are needed (400-1000). For high quality CMYK profiles, 1000-3000 is not an unreasonable number of patches.
How many patches are on commercially available target charts?
Most commercially available target charts have a relatively low number of color patches and are made with a limited number of pigments (do read the whole linked-to thread — it's very informative; Graham Gill did later add the "-am" simple linear gamma matrix profiling option). When using a commercially available target chart, you will get a more useable camera input profile, with less chance of profile-induced "curve fitting" artifacts, by making a matrix profile (-am, -ag, -aG, -as, -aS).
Unless you have a good reason to think that your camera really does not respond linearly to light, you will get a more useable general purpose profile by making a simple linear gamma matrix profile ("colprof -am").
Making LUT profiles for capturing scene-specific colors under controlled studio lighting
Nothing about cameras would indicate that when making a LUT profile for a camera you somehow need fewer color patches than for making a LUT profile for a monitor or printer, and in fact you might need a whole lot more patches, depending on the scenes you intend to photograph:
- On the one hand, for printer and monitor profiles, the color patches are generated by the device that will be profiled, so the colorants/pigments in the color patches are guaranteed to be the colorants/pigments produced by the device itself.
- On the other hand, when profiling a camera, ideally and for optimal results the pigments used in the target chart should be representative of the pigments in all the scenes that will be photographed and interpolated using the resulting profile, and also should evenly cover the color gamut of said scenes. To say the least, this is a "tall order" that actually can't be fulfilled except in the limiting case of custom made target charts composed of pigments obtained from the objects being photographed, under controlled studio lighting.
If you are photographing artwork or ceramic tiles or paint samples or whatever other "limited by design" set of colors in your studio, with predictable and controlled lighting, then you can make a custom target chart covering exactly the pigments in the objects being photographed, and you can make a LUT profile that will do an excellent job of making photographs of those objects match what the eye sees when viewing the scene.
Even if you don't go so far as to make a custom target chart, you can still make a decent controlled lighting LUT camera input profile using a target chart with a lot of patches. If you want good advice on making custom target charts, and on making and using LUT camera input profiles under controlled studio lighting (that is to say, one profile per type of lighting, with the lighting itself being of sufficiently wide spectrum and high quality), do search the ArgyllCMS mailing archives. Better yet, search the archives and also subscribe to the ArgyllCMS mailing list — people on the list are both knowledgeable and friendly to newbies seeking advice.
No, DCP (Dng Camera Profiles) "profiles" are not better than ICC matrix camera profiles
Sometimes you'll run across the claim that Adobe's DCP profiles are Look Up Table ("LUT") profiles and therefore DCP profiles are better than ICC matrix camera profiles that you might make using ArgyllCMS or other camera profiling software. This claim is based on the blanket conclusion that "ICC LUT profiles are better than ICC matrix profiles", combined with some conceptual confusion about what a DCP profile really is:
- Blanket conclusion: "Better than" implies "better than for what purpose". For a camera input profile made using a commercially available target chart, for use under a wide variety of scenes and lighting, and presuming your camera does respond linearly to light, a simple linear gamma matrix profile is the better choice.
- Conceptual confusion: The DCP profile assigns a simple linear gamma ICC matrix profile to the interpolated camera raw file, and then converts the RGB data to the ProPhotoRGB color space, and then applies an "HSV" LookUp Table to the ProPhotoRGB colors to make them prettier, or to emulate various "picture style" as used by various camera manufacturers, or etc. If what you really want is linearized RGB data proportional to the scene you actually photographed, DCP profiles aren't the way to get there.
You might or might not like what DCP profiles do to the colors in your interpolated camera raw files. But the proper comparison is not between the DCP "HSV" LUT and a linear gamma matrix camera profile that you might make using ArgyllCMS. Rather the proper comparison is between the linear gamma matrix profile that's at the start of the DCP profile image processing chain (often the same matrix that is constructed from the dcraw "adobe_coeff" table), and a corresponding linear gamma matrix profile you might make with ArgyllCMS. See Use Dng Camera Profiles (DCP)? for a short summary of and links to more information about the limitations of DCP profiles.
Note to digiKam users: The digiKam/showFoto raw processor produces interpolated RGB data that's been "gamma corrected" (this is true for digiKam through version 4.7; I haven't checked more recent versions) to have the default dcraw curve. The dcraw default curve is not a true gamma curve but rather somewhat close to the sRGB TRC. So if you are making an ArgyllCMS camera input profile for use with digiKam, try the colprof "-aG" or "-aS" options (neither options are likely to exactly reproduce the default dcraw "gamma correction", but they'll both be close). The colprof "-am" option presumes the interpolated RGB data is linear, and of course "gamma correcting" the interpolated RGB data during raw processing destroys that presumed linearity. If you want linear gamma RGB data out of digiKam, file a bug report.
Compensating for camera veiling flare
Camera veiling flare
A simple linear gamma profile ("colprof -am") looks for matrix primaries that will provide the closest possible match between the target chart shot color patch RGB values and the corresponding XYZ colors given in the target chart reference file. This type of profile doesn't do any tone curve correction and simply assumes that the camera sensor's response to light is linear.
In point of fact, camera veiling flare pretty much guarantees that the sensor's response to light isn't really linearly proportional to the amount of light entering the lens from the scene that's being photographed. I'm not talking about the more obvious type of lens flare that can be eliminated by using a lens hood or pointing the camera away from the scene light sources. Rather I'm talking about light getting into the lens or camera and randomly bouncing around before hitting the sensor.
This "random light bouncing" might be from reflections off of dust particles in the lens or in the camera, or off the inside of the camera body, or off the surfaces of the various optical elements in the lens and any lens filter you might be using, or from any number of other sources. The resulting "camera veiling flare" is equivalent to adding a certain fixed amount of exposure to all portions of the sensor, as if another light source had evenly added a little bit of light to every item in the scene being photographed.
Why making a shaper matrix or LUT profile that automatically compensates for camera flare isn't a such a good idea for general purpose camera profiles
Unlike a simple linear gamma matrix profile, the shaper matrix and LUT profiles both assume that the camera's response to light coming off the scene being photographed might not be linear. Camera veiling flare (at least partially) accounts for the fact that shaper matrix and LUT profiles tend to produce darker shadows than a simple linear gamma matrix profile. However, for a general purpose camera input profile the better way to account for camera veiling glare is to put a black trap (aka "light trap") in the target chart shot. Here's why:
- Given that most sources of camera veiling flare are equivalent to adding a certain fixed amount of exposure to all portions of the sensor, the percent increase in exposure is greatest in the shadows of an image and least in the highlights. Unfortunately, affordable commercially produced target charts tend to not have a whole lot of really dark color patches.
- Even more unfortunately, in the target chart reference file the darker colors tend to have higher expected errors, and as recorded by the camera, darker color patches have a higher signal to noise ratio. So just where you need more precision and more patches to accurately model camera flare, you actually have less precision and fewer patches. Consequently any bends and twists in the shadows of shaper matrix and LUT profiles made using commercially available target shots are just as much a product sensor noise and low precision color patch values as they are of actual camera flare.
By its very nature, camera flare varies from lens to lens, scene to scene, the amount of dust inside the camera and lens, and for various other more or less unpredictable and uncontrollable reasons. So for those of us who want a general purpose camera input profile, there is no such thing as "just the right amount of compensation for camera flare". But there is an easy "good enough" solution: Put a black trap in the target shot. A black trap will at least get you in the right ballpark, without having to worry about "how many dark color patches, with how much measurement precision in the reference file".
Summary and notes
This article is somewhat long and no doubt covers too much territory, but it can be summarized pretty succinctly:
- To compensate for camera veiling glare, put a black trap in your target chart shot.
- Don't get stuck on trying to make a profile with the lowest possible deltas. Instead make a profile that suits your purposes and the limitations of your target chart:
- For a general purpose camera input profile that's made using a commercially available target chart, make a matrix profile. Make a simple linear gamma matrix profile unless you have a really good reason to make a different type of matrix profile.
- For specialized photography under controlled studio lighting, where you need the photograph to match the objects being photographed as closely as possible, find or make an appropriate target chart, preferably with pigments representative of the objects you are actually photographing, and make a LUT profile.
- Check your profile's TRCs and xicclu graphs to see if the profile exhibits unexpected twists and channel crossovers.
More than one general purpose profile per camera?
You can make a separate general purpose camera input profile for each of your lenses, to control for lens-to-lens variations in light scattering and for color casts that might be lens-specific. You can also make separate general purpose profiles for each lens+filter combination that you use.
Personally, I only have two cameras (a Canon 400D and a Sony A7), I only use two lenses (prime lenses that fit both cameras using lens adapters), and I seldom use filters with my lenses. So making a separate profile for each lens, for each camera, would be easy enough to do. But the type of photography that I do really doesn't require that my general purpose camera input profiles be super accurate in the shadows. Or else I'm just too lazy to make four camera input profiles instead of just two. Take your pick.
You also can make simple linear gamma camera profiles for use in different types of full spectrum light, such as shade, daylight, direct sunlight, tungsten light, and etc. But again, weigh the aggravation against how much benefit you might get from making all those camera input profiles. If you experiment and you decide it really is worth the effort to make multiple lighting- and lens-specific general purpose camera input profiles, drop me an email — I'd be interested in knowing how many profiles you made, under what lighting conditions, and where it makes a discernible difference in your photographs.
Notes on the camera, profiles, and raw processer used in this article:
The sample images were shot raw using a Canon 400D/Rebel xti camera.
The ArgyllCMS camera input profiles were made in 2012 or before, using whatever version of ArygllCMS was current at the time. The Canon 400D/xti Adobe Dng profile was probably obtained using DCP to ICC, but frankly I obtained/produced this profile so many years ago that I don't remember when, where, or how. The colors it produces do match the colors that are produced when I use the Adobe Dng profile that comes with dcraw. You can find the raw numbers for constructing Adobe Dng profiles in the dcraw source code "adobe-coeff" table.
The image raw files were processed in RawTherapee (the version of RawTherapee that was current in December 2012), using the following parameters:
- The RawTherapee neutral post-processing profile.
- DCB interpolation (0 iterations, no false color suppression, no post demosaic artifact/noise reduction; today I mostly use AMAZE interpolation).
- "Blend highlights with matrix" was disabled.
- No other image processing was done, except adjusting the raw white point to bring all output images to the same white point as measured at a preselected point in each image. As explained above, all profiles except the Adobe Dng and the ArgyllCMS simple linear gamma matrix profiles did require that the image be given varying amounts of negative exposure compensation to avoid blowing out the image highlights.
- WideGamutRGB as the working and the output profile (frankly I don't remember why I chose WideGamutRGB rather than some other color space; any of the larger standard working spaces would have worked equally well).