How to Make a Better Custom Camera Input Profile
This article shows how to use ArgyllCMS and a target chart shot to make a general purpose custom camera input profile that is:
- Color balanced, so it won't add a color cast to your interpolated raw files.
- Normalized, so it won't make your interpolated raw files darker or lighter than they really are.
- Well behaved, so it can be used as an RGB working space for editing your interpolated raw files.
Written August 2014. Minor update in February 2015.
Introduction: Making a better general purpose custom camera input profile
This article shows how to use ArgyllCMS to make a general purpose (simple linear gamma matrix) custom camera input profile that is color balanced and normalized — two very desirable characteristics in a camera input profile. The resulting profile will also be well behaved and so can be used as a working space when editing your interpolated camera raw files.
If you don't want a general purpose (simple linear gamma matrix) camera input profile, but instead are making a profile for controlled studio lighting (for example a LUT profile or a shaper-matrix profile), the directions on this page will help you make a better controlled studio lighting input profile. All you need to do is change the colprof parameters used to make the final camera input profile, to the appropriate parameters for making the type of camera input profile that suits your needs (for example from "colprof -am -u" to "colprof -ax -u"). The ti3 "pre-processing" steps are the same regardless of what kind of profile you want to make.
I posted a "mini how to" on the ArgyllCMS mailing list back in December, 2012, that worked with ArgyllCMS version 1.4: Re: Profile input white not mapping to output white. However, ArgyllCMS version 1.5 changed the way the colprof "-u" switch works. So here are updated instructions that work with ArgyllCMS versions 1.50 through 1.6.3 (and hopefully beyond). The updated procedure is easier and as a bonus produces lower maximum and average errors.
I can't guarantee that the procedure described on this page will work with every target shot made by every camera, as processed by every raw processor, but it seems pretty robust. I've tested the procedure using:
- Target shots made with the following cameras: Canon 400D, Nikon D700, Pentax K5_II, Sony A7, and Sony Nex-7.
- The Darktable, dcraw, Photivo, RawTherapee, and UFRaw raw processors. I didn't test all five raw processors on target shots for all five cameras (that would be a big task). But I did test all five raw processors on Pentax K5-II and Sony A7 target shots, and all the raw processors produced useable input profiles.
I haven't tested the following procedure on cameras that don't use Bayer filters. Nor have I tested it with target charts other than Wolf Faust's 288-patch IT8 target charts.
Note to digiKam users: The digiKam/showFoto raw processor produces interpolated RGB data that's been "gamma corrected" to have the default dcraw curve (this is true for digiKam through version 4.7 — I haven't checked more recent versions). Even when you ask for raw color output, the digiKam raw processor applies a non-linear TRC (roughly gamma=2.2) to the resulting interpolated raw file. The procedure on this page assumes that the raw processor outputs linear gamma results, that is, interpolated without applying any TRC. At the very least you'd need to use the colprof profiling parameter "-aS" or "-aG" instead of "-am" as recommended by the procedure on this page, but I haven't tested this procedure with the digiKam raw processor.
Note to people with cameras that capture an HDR image by automatically compressing and tone mapping the sensor data: This procedure assumes your camera doesn't applying tone mapping to the information captured by the sensor. So the procedure won't work with cameras (or camera settings, if this option can be enabled/disabled) that tone map the information saved in the raw file. The procedure on this page does require that the information in the raw file is proportional to the light coming from the scene that was photographed ("scene linear").
Using ArgyllCMS to make a camera input profile
For a step-by-step "how to" on making a good target chart shot, see How to Make a Camera Input Profile Target Shot.
Assuming you've already made a decent target shot, the basic ArgyllCMS camera profiling procedure is pretty straightforward:
- Raw process the target chart shot raw file to produce a 16-bit linear gamma raw color tiff. Disable all post-interpolation enhancement and also all noise reduction algorithms.
- Use the ArgyllCMS scanin utility to read the RGB color patch values in the target chart tiff. Scanin needs the target chart tiff and also the reference file that supplies the XYZ values for the target chart color patches. Scanin stores the color patch RGB values and the reference chart information in a special file called a "ti3" file, which is saved to disk as "name_of_target_chart_tiff.ti3".
- Use the ArgyllCMS colprof utility to make a profile using the ti3 file saved by scanin. Colprof uses the scanin-read color patch RGB values in the ti3 file to do (something like) a least squares analysis to produce an input profile that makes all the target shot color patch RGB values match as closely as possible to the corresponding XYZ color patch values that were given in the target chart reference file.
When raw processing the target shot, the exposure compensation and white balance need to be just right
Colprof works with the color patch RGB values that scanin reads from your interpolated target chart tiff. So assuming you've made a good target shot, the major obstacle to making a good camera input profile is getting the exposure compensation and color balance just right during raw processing. (If you followed my "how to" on making a camera profile target chart shot, you might be jumping up and down saying "But why didn't you mention this???" The answer is, I didn't need to — the procedure on this page takes away the guess work.)
A couple of examples will illustrate the sort of unhappy results that can be expected if you feed scanin a target shot tiff that wasn't perfectly color balanced or was given the wrong exposure compensation during raw processing:
- If the target chart was color balanced to be too yellow and was given too much negative exposure compensation . . . In this case scanin will read RGB values that are too yellow and underexposed. And colprof will produce a profile that adds blue to compensate for the yellow color cast, and also adds positive exposure compensation to compensate for the overexposed target shot tiff color patches.
When applied to the target shot, the target shot will look great. But if this same profile is applied to an interpolated raw file that already has the correct exposure and color balance, the resulting image will be too blue and also too bright. The "too bright" result can be compensated for by using the colprof "-u" or "-U" options, but the "blue color cast" result cannot.
- If the target chart was color balanced to be too green and was given too much positive exposure compensation . . . In this case scanin will read RGB color patch values that are too green and overexposed, and colprof will produce a profile that adds magenta to compensate for the green color cast, and also adds negative exposure compensation to compensate for the overexposed target shot tiff color patches.
The resulting camera input profile will make the target shot itself look great. But if this same profile is applied to an interpolated raw file that already has the correct exposure and color balance, the resulting image will have a magenta cast and also be too dark. I'm not sure whether using the colprof "-u" option helps with overexposed target shots, though using the "-U" option would help.
Therefore making the best possible camera input profile — one that doesn't do any exposure compensation and doesn't "correct" nonexistent color casts in already properly color balanced images — requires that you:
- Expose the target chart using exactly the right in-camera exposure, or else use exactly the right exposure compensation during raw processing.
- Use exactly the right white balance during raw processing.
This is why camera profiling "how tos" very rightly spend a great deal of effort explaining how to properly expose and white balance your target shot. The farther away from "just right" the raw-processed target chart tiff's exposure and color balance are, the worse the resulting camera input profile.
Why colprof probably won't agree with your chosen white balance and exposure compensation
On the one hand, it really is important to get the in-camera exposure as close to perfect as possible, because doing so ensures that the target data isn't unnecessarily noisy from being underexposed and also that the brightest patches aren't blown from being too far overexposed.
On the other hand, no matter how hard you try to get the exposure compensation and white balance just right during raw processing, colprof will almost certainly come up with a different amount of exposure compensation and a different color balance. This is true for three reasons:
- Colprof necessarily assumes that the brightest more or less neutral color patch on the target chart represents "media white". This assumption is true when profiling scanners, printers, and monitors. However, "media white" for a camera has nothing to do with the color of the brightest more or less neutral color patch on the target chart. Rather "media white" for a camera is determined by the light illuminating the target. To measure "media white", ideally you would put a 100% diffusely reflecting Lambertian reflector in the target shot. Near-perfect Lambertian reflectors do exist, but including them as part of a commercially available target chart would raise the cost quite a bit (though you can add your own very simple DIY near-perfect Lambertian reflector to your target chart shot).
- Colprof necessarily assumes that the darkest more or less neutral color patch on the target chart represents "media black". Again, this assumption is true when profiling scanners, printers, and monitors. However, "media black" for a camera theoretically is solid black, which would be the color of a perfect black trap, though camera flare complicates the situation. Near-perfect DIY black traps can easily be made and added to your target shot, but commercially available target charts don't include black traps.
- Colprof does a kind of least squares analysis of the color patch values produced by scanin to make a profile that provides the best fit to all of the target chart color patches. There's no way to predict in advance of actually making the profile exactly what color balance and exposure compensation are required to produce that best fit.
How to convince colprof to use the right exposure compensation and white balance
The solution to the problems posed by profiling cameras using commercially available target charts is likewise three-fold:
- When raw processing the target shot, set the white balance using either the most neutral color patch on the target chart itself or by color balancing using a known neutral object that was included in the target chart frame. If the known neutral object is a near-perfect Lambertian reflector, so much the better.
- Use values in the target chart reference file to set the target chart white and black point, either during raw processing or by opening the interpolated raw file with an external high bit depth image editor. If you included a DIY (Do It Yourself) 100% reflective near-perfect Lambertian reflector and/or black trap in the target shot, these also can be used to help set the target chart white and black points.
- Add synthetic D50 white and black color patches to the ti3 file produced by scanin, so that colprof picks the synthetic white and black color patches as the profile white and black points.
Step-by-step procedure for making a better camera input profile
Here's the step-by-step procedure for making a better camera input profile that can also be used as a well behaved RGB working space.
- While raw processing the target chart, set the target shot white balance. Either use the most neutral target chart color patch to set the white balance. Or even better, put a known neutral object in the target chart frame and use it for white balancing the image.
- Either during raw processing or in an external high bit depth image editor using Levels (RGB-based, not LAB-based! there's no way to convert the interpolated target chart to CIELAB without a camera input profile!), use the information in the target chart reference file to set the target shot white and black points. If you included a near-perfect 100% diffusely reflecting Lambertian reflector, and/or a black trap in the target shot frame, they also can be used to help set the target chart white and black points.
- Crop the target shot to exclude everything except the actual target chart.
- Use scanin to create the ti3 file.
- Open the ti3 file and add perfect synthetic D50 white and black color patches.
- Use colprof and the modified ti3 file to make the profile. Colprof will pick the synthetic D50 white and black patches as the profile white and black points. If you are making a general purpose linear gamma matrix profile (colprof options "-am -u"), the resulting profile will be normalized, color balanced, and well-behaved.
- Use xicclu to verify that the resulting camera input profile is also a well-behaved working space profile.
Below is a worked example showing the individual camera profiling steps for a Pentax K5-II target chart shot. The links in the procedure given above lead to the corresponding step in the worked example given below.
The actual camera profiling procedure is very simple and quick to do, and it will take you longer to read through the worked example than to actually make yourself a better general purpose camera input profile:
Worked example for a Pentax K5-II camera
The owner of the Pentax K5-II sent me the target shot raw file and kindly gave permission to use it in this tutorial. The Wolfe IT8 target shot "Daylight.PEF" was made in direct sunlight. The target chart itself covers the middle three-quarters of the frame and I will guess that the lens was stopped down, so lens vignetting shouldn't be a problem. The target chart is mounted on a dark gray surround to minimize glare and veiling from reflected light:
While raw processing the target chart, set the target shot white balance
Determining the most neutral target chart color patch
The target shot doesn't include a known neutral object for white balancing, so the first step is to determine which target chart color patch is closest to being completely neutral. The Wolf Faust IT8 target chart includes two sets of neutral color patches:
- Column 16, color patches A16 through L16
- A row of GrayScale color patches at the bottom of the target chart, labelled "GS##". The brightest patch at the extreme left is referred to as GS0 and the darkest patch at the extreme right is referred to as GS23.
To determine which neutral color patch is the closest to being completely neutral, open the manufacturer-supplied reference file, which for this target chart is R131007.txt. There are some preliminary lines of information plus thirteen columns of data, that look like this:
IT8.7/2 ORIGINATOR "Wolf Faust" DESCRIPTOR "L* a* b* Batch average data (light D50, viewing angle 2)" MANUFACTURER "Wolf Faust - http://www.coloraid.de" CREATED "October 10, 2013" PROD_DATE "2013:10" SAMPLE_BACKING "white" SERIAL "R131007" MATERIAL "Kodak Professional Endura Premier" KEYWORD "MEAN_DE" # Mean Delta E of samples compared to batch average NUMBER_OF_FIELDS 13 BEGIN_DATA_FORMAT SAMPLE_ID XYZ_X XYZ_Y XYZ_Z LAB_L LAB_A LAB_B LAB_C LAB_H D_RED D_GREEN D_BLUE D_VIS END_DATA_FORMAT NUMBER_OF_SETS 288 BEGIN_DATA A1 3.40 2.83 2.00 19.37 11.54 3.06 11.95 14.83 1.36 1.66 1.60 1.51 A2 4.25 2.87 1.65 19.50 23.57 6.95 24.57 16.42 1.21 1.83 1.69 1.47 A3 5.11 2.94 1.43 19.78 33.53 9.95 34.98 16.53 1.10 2.02 1.76 1.42 A4 5.61 3.19 1.55 20.77 35.27 10.28 36.74 16.25 1.06 2.01 1.72 1.39 . . . GS21 0.94 1.00 0.84 8.97 -0.79 -0.37 0.88 205.57 2.03 1.98 1.97 2.00 GS22 0.76 0.80 0.68 7.24 -0.53 -0.44 0.69 219.08 2.13 2.08 2.06 2.10 GS23 0.42 0.50 0.54 4.51 -2.32 -2.36 3.31 225.47 2.47 2.23 2.16 2.33 END_DATA
To figure out which color patches are closest to being completely neutral, you only care about the fifth, sixth, and seventh columns that are labelled LAB_L, LAB_A, and LAB_B. It's also a good idea to only look at color patches that have LAB_L values that are greater than 50, because the signal-to-noise ratio decreases as the color patches get darker.
A completely neutral color patch would have LAB_A and LAB_B both equal to 0.00. So locate the color patch that has the smallest LAB_A and LAB_B values and a LAB_L value that's at least 50.00. It will either be one of the color patches in Column 16 or else one of the color patches in the GrayScale along the bottom of the chart.
Here are the candidates for the most neutral color patch from the R131007 reference file:
SAMPLE_ID XYZ_X XYZ_Y XYZ_Z LAB_L LAB_A LAB_B A16 74.96 78.15 63.87 90.85 -0.81 0.59 B16 59.10 61.64 51.35 82.72 -0.62 -0.54 C16 46.29 47.93 39.55 74.78 0.22 0.01 D16 35.67 36.86 30.65 67.17 0.44 -0.37 E16 26.77 27.71 23.02 59.63 0.19 -0.28 F16 19.28 19.96 16.64 51.79 0.18 -0.39 GS0 77.17 80.83 64.71 92.05 -1.52 1.87 GS1 66.38 69.14 57.40 86.57 -0.62 -0.35 GS2 58.35 60.79 50.57 82.26 -0.64 -0.46 GS3 51.28 53.21 44.75 78.00 -0.08 -1.02 GS4 45.09 46.82 39.25 74.07 -0.15 -0.82 GS5 39.51 40.90 34.40 70.11 0.22 -0.94 GS6 34.38 35.57 29.73 66.19 0.29 -0.61 GS7 29.91 31.03 25.95 62.53 -0.03 -0.6 GS8 25.34 26.35 21.80 58.36 -0.25 -0.12 GS9 21.40 22.16 18.44 54.20 0.12 -0.34 GS10 17.97 18.63 15.37 50.25 0.04 0.01
Technically what you want is the color patch with the smallest "square root of the sum of the squares of LAB_A and LAB_B". But you can tell which color patch is closest to being completely neutral just by looking for the patch with the smallest absolute values (ignore the signs) for LAB_A and LAB_B. If in doubt, set up a spreadsheet and do the calculations.
Looking at the candidates for the most neutral color patch with a LAB_L value that's greater than 50.00, the most neutral patch turns out to be GS10, with LAB_A=0.04 and LAB_B=0.01. The second most neutral is GS07, with LAB_A=-0.3 and LAB_B=-0.6. I used color patch GS10 to white balance the target shot.
The most neutral color patch does vary from one IT8 target chart to another, except of course for batch-measured target charts that have the same Charge.
When raw processing the target shot, make sure all image enhancing and noise-reduction algorithms are disabled. With RawTherapee:
- Under "Processing Profiles", select the Neutral preset.
- On the "Raw" tab, select your favorite interpolation algorithm and leave all the other options at the default Neutral settings. I selected the amaze algorithm and left "Link greens" checked.
It might be worth experimenting with raw interpolation options such as black frame subtraction, hot pixel removal (this one might be good to use anyway if you know your camera has hot pixels, but scanin does have provisions for removing outliers), and etc. But as a baseline for comparison, make a profile without using such options.
- On the "Color" tab, "Spot WB" using the most neutral target chart color patch, or better yet on a known neutral object that was included in the target shot. There was no known neutral object included in the target shot, so I spot white balanced on the most neutral target chart color patch, which is GS10.
- Then select "Save Reference Image for Profiling" and save the interpolated target shot to disk as a 16-bit integer tiff. I save the target chart under the name "Pentax_K5-II_Sunlight.tiff".
Profiling a camera requires that the interpolated target shot be output as "raw color". What this means is that no ICC profile conversion has been done that converts the interpolated target shot from a camera input profile to an RGB working space. Assigning a profile doesn't change the raw color RGB values. But assigning a profile and then converting to some other profile does change the RGB values, at which point you no longer have raw color.
In RawTherapee, selecting "Save Reference Image for Profiling" bypasses all ICC profile conversions and also all post-interpolation target shot processing.
Set the target shot white and black points
Use the target chart reference file to determine the target shot white and black points:
The target chart reference file tells you the XYZ values for the brightest (GS00) and darkest (GS23) GrayScale color patches. And you know that in a properly color-balanced target shot, if the target shot RGB values and the reference file XYZ values are on the same scale, then the RGB Green channel values for these nearly neutral color patches should be approximately equal to the corresponding reference file XYZ Y values (well, if you didn't already know this, now you do!).
The reference file XYZ Y channel values are on a scale from Y=0 for solid black to Y=100 for solid white. So to put the RGB and XYZ values on the same scale:
- If your image editor or raw processor color picker reads out RGB values on a scale from 0 to 255, multiply the relevant target chart Y values by 2.5 (or divide the RGB values by 2.5, your choice).
- If your image editor or raw processor color picker reads out RGB values on a scale from 0 to 65535, multiply the relevant target chart Y values by 655.35 (or divide the RGB values by 655.35).
- If your image editor or raw processor color picker reads out RGB values on a scale from 0.0 to 1.00, divide the relevant target chart Y values by 100 (or multiply the RGB values by 100).
The box below shows the reference file GS0 and GS23 XYZ Y channel values scaled to 16-bit integer (highlighted in green) so the Y channel values can be used to set the white and black points for the RawTherapee-interpolated 16-bit integer Pentax K5-II target shot tiff:
SAMPLE_ID XYZ_X XYZ_Y XYZ_Z Y (255) Y (65535) Y (1.00) GS0 77.17 80.83 64.71 206 52972 0.81 GS23 0.42 0.50 0.54 1 328 0.01
Some raw processors can be used to set the target shot white and black points, although it's not always easy to do so very precisely. RawTherapee has a nice color picker readout. But the RGB values that it reads aren't raw color RGB values but rather are based on the "Color" tab "Working Profile". This means the color-picked RGB values are not raw but rather have already been converted from the designated Input Profile (even though we are planning on outputting raw color) to the designated Working profile and on top of that, are gamma-corrected (eg Gamma=1.8 for ProPhoto, 2.2 for WideGamut, and etc).
So for RawTherapee-interpolated target shots, setting the target shot white and black points needs to be done in an external image editor. I opened the RawTherapee-processed target shot tiff "Pentax_K5-II_Sunlight.tiff" using a version of high bit depth GIMP 2.9 from git that I patched to eliminate a whole bunch of problems with the default GIMP 2.9 (which is in a state of flux, being a development version). In GIMP I assigned a linear gamma version of the sRGB ICC profile (the default-assigned regular sRGB profile makes the target chart look very dark). Again, assigning an ICC profile doesn't modify the RGB values, so even though an ICC profile has been assigned, it's still a raw color image.
Keeping the bit depth at 16-bit integer, I used the GIMP Levels Low and High Value channel sliders (which actually moves all three channels together by the same amount) to set the target shot white and black points, getting the GS0 and GS23 RGB "Green" channel values reasonably close to the corresponding "Y values multiplied by 655.35". Don't adjust the individual Red, Green, or Blue channel sliders or you'll change the target shot white balance that you just worked so hard to set!
Figure 3 below shows the Levels adjustment and resulting GS0 and GS23 RGB channel values:
Crop the target shot
Sometimes the background surrounding the target chart confuses scanin. Sometimes even an almost completely featureless background can confuse scanin (oddly enough, sometimes depending on which raw processor was used to process the target chart raw file). So as shown in Figure 4 below, it's always a good idea to crop the target chart before using scanin to read in the target chart color patch RGB values:
After using Levels to set the white and black points and cropping the target chart tiff to include just the target chart, I exported the image as "Pentax_K5-II_Sunlight-cropped.tif". If you are using GIMP 2.9 from git to adjust and crop and the target chart tiff, when you export the modified image file, don't use compression, or else scanin will choke on the tiff. If you don't disable writing metadata (exif, xmp, etc) and so forth, scanin might print a string of warnings to screen, but the warnings should be harmless (if not, filing a bug report with GIMP might be a good idea).
Use scanin to create the ti3 file
Here's the terminal command and output for generating the ti3 file from the cropped target chart tiff "Pentax_K5-II_Sunlight-cropped.tif", the "it8.cht" chart recognition template, and the "R131007.txt" reference file:
$ scanin -dipn -G1.0 -p Pentax_K5-II_Sunlight-cropped.tif it8.cht R131007.txt 1 extra (alpha ?) samples will be ignored Perspective correction factors = 0.000001 -0.000003 1666.500000 1180.000000 Writing output values to file 'Pentax_K5-II_Sunlight-cropped.ti3'
The scanin warning about the "extra (alpha ?) samples" is harmless and can be ignored.
Below is a brief explanation of what the various scanin command line options mean:
- -dipn : output a diagnostic tif to confirm that scanin correctly identified the color patches. Having scanin output a diagnostic jpeg file is always a good idea.
- -G1.0 : assume the target shot was interpolated and output using a linear tone curve.
- -p : compensate for any perspective distortion.
- Pentax_K5-II_Sunlight-cropped.tif : the name of the raw color target shot tiff whose interpolated raw color RGB values scanin will read in.
- it8.cht : the template scanin will use to read the target shot tiff. Target shot template files are supplied by ArgyllCMS.
- R131007.txt : the name of the reference file that came with the camera profiling target. If you have a Wolf Faust IT8 target, you want to use the reference file with a file name like "Rxxxxxx.txt".
For more information about scanin command line options, see the scanin documentation.
Here's what the first few lines of the "Pentax_K5-II_Sunlight-cropped.ti3" file look like:
CTI3 DESCRIPTOR "Argyll Calibration Target chart information 3" ORIGINATOR "Argyll target" CREATED "Thu Aug 28 15:43:32 2014" KEYWORD "DEVICE_CLASS" DEVICE_CLASS "INPUT" KEYWORD "COLOR_REP" COLOR_REP "XYZ_RGB" KEYWORD "STDEV_R" KEYWORD "STDEV_G" NUMBER_OF_FIELDS 10 BEGIN_DATA_FORMAT SAMPLE_ID XYZ_X XYZ_Y XYZ_Z RGB_R RGB_G RGB_B STDEV_R STDEV_G STDEV_B END_DATA_FORMAT NUMBER_OF_SETS 288 BEGIN_DATA A01 3.40000 2.83000 2.00000 4.05637 2.52746 2.88878 0.158385 0.100542 0.114722 A02 4.25000 2.87000 1.65000 5.61413 2.01413 2.34763 0.177354 0.0911968 0.104855 A03 5.11000 2.94000 1.43000 7.23068 1.62348 1.98254 0.209020 0.0878116 0.100149
In the excerpts from the ti3 file shown above, the values highlighted in yellow are the XYZ values from the "R131007.txt" reference file, and the values highlighted in green are the RGB values read in by scanin.
Add perfect synthetic D50 white and black color patches to the ti3 file
We don't want colprof to use any of the target chart color patches to set the media white and black points. Rather we want colprof to use D50 white as the media white point, and solid black as the media black point. When using a scale from 0 to 100, D50 white has the XYZ values (96.4200, 100.000, 82.4910), and solid black has the XYZ values (0.0, 0.0, 0.0). So we'll add two lines to the ti3 file:
00W 96.4200 100.000 82.4910 100.000 100.000 100.000 0.000000 0.000000 0.000000 00B 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.000000 0.000000 0.000000
The first line tells scanin that the target chart and reference file had a color patch for D50 white and that the corresponding target chart tiff RGB values were (100.000 100.000 100.000). The second line tells scanin that the target chart and reference file had a color patch for solid black and that the corresponding target chart tiff color patch RGB values were (0.00000, 0.00000, 0.00000).
The modified ti3 file will have 290 instead of only 288 lines. So the "NUMBER_OF_SETS" also needs to be changed from 288 to 290. I saved the modified ti3 file under the name "Pentax_K5-II_Sunlight-cropped-addbw.ti3", the first few lines of which look like this:
CTI3 DESCRIPTOR "Argyll Calibration Target chart information 3" ORIGINATOR "Argyll target" CREATED "Thu Aug 28 15:43:32 2014" KEYWORD "DEVICE_CLASS" DEVICE_CLASS "INPUT" KEYWORD "COLOR_REP" COLOR_REP "XYZ_RGB" KEYWORD "STDEV_R" KEYWORD "STDEV_G" NUMBER_OF_FIELDS 10 BEGIN_DATA_FORMAT SAMPLE_ID XYZ_X XYZ_Y XYZ_Z RGB_R RGB_G RGB_B STDEV_R STDEV_G STDEV_B END_DATA_FORMAT NUMBER_OF_SETS 290 BEGIN_DATA 00W 96.4200 100.000 82.4910 100.000 100.000 100.000 0.000000 0.000000 0.000000 00B 0.000000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.000000 0.000000 A01 3.40000 2.83000 2.00000 4.05637 2.52746 2.88878 0.158385 0.100542 0.114722 A02 4.25000 2.87000 1.65000 5.61413 2.01413 2.34763 0.177354 0.0911968 0.104855 A03 5.11000 2.94000 1.43000 7.23068 1.62348 1.98254 0.209020 0.0878116 0.100149
Use colprof and the modified ti3 file to make the profile
The code box below shows the colprof command line used to make a general purpose camera input profile from the "Pentax_K5-II_Sunlight-cropped-addbw.ti3" file, plus the resulting terminal ouput. Note that the colprof command requires that you not add the extension "ti3" to the name of the ti3 file from which the profile is made:
$ colprof -v -am -u -C"No copyright, use freely." -O"Pentax_K5-II_Sunlight.icc" \ -D"Pentax_K5-II_Sunlight.icc" Pentax_K5-II_Sunlight-cropped-addbw No of test patches = 290 Find white & black points Picked white patch 2 with dev = 1.000000 1.000000 1.000000 XYZ = 0.964200 1.000000 0.824910, Lab = 100.000000 0.000000 -0.000808 Picked black patch 1 with dev = 0.000000 0.000000 0.000000 XYZ = 0.000000 0.000000 0.000000, Lab = 0.000000 0.000000 0.000000 Creating matrix... 100% Matrix = 0.643434 0.249044 0.034023 0.250610 0.904516 -0.198552 0.005734 -0.140864 0.898335 Matrix = 0.643434 0.249044 0.034023 0.250610 0.904516 -0.198552 0.005734 -0.140864 0.898335 Doing White point fine tune: Before fine tune, rel WP = XYZ 0.926501 0.956574 0.763205, Lab 98.295951 0.741822 2.177784 After fine tune, rel WP = XYZ 0.964200 1.000000 0.824900, Lab 100.000000 0.000000 0.000000 abs WP = XYZ 0.926501 0.956574 0.763214, Lab 98.295950 0.741815 2.176995 Black point XYZ = 0.000000 0.000000 0.000000, Lab = 0.000000 0.000000 0.000000 White point XYZ = 0.926501 0.956574 0.763214 Black point XYZ = 0.000000 0.000000 0.000000 Done gamma/shaper and matrix creation Profile done Profile check complete, peak err = 14.605176, avg err = 2.634383
Figure 2 above shows the interpolated, cropped Pentax K5-II target shot after applying the Pentax_K5-II_Sunlight.icc camera input profile. Below is a brief explanation of what the various colprof command line options mean:
- -v : be verbal
- -am : make a simple linear gamma matrix profile
- -u : quoting the colprof documentation, "Using the -u flag causes the media white point to be automatically scaled . . . to avoid clipping values up to full device white".
In general it's a good idea to use the "-u" flag when making a camera input profile. Otherwise colprof assumes the brightest patch on the target is the brightest object you'll ever photograph (if you were profiling a printer instead of a camera, the brightest patch on the printer test chart is probably paper white, which usually really is the brightest color you can print). However, in the present case this command line option won't make any difference at all.
- -C"No copyright, use freely." : copyright information.
- -O"Pentax_K5-II_Sunlight.icc" : the file name of the camera input profile that colprof will produce. The default file name is just the base name of the ti3 file plus the extension ".icc". You could add information about what parameters were used to make the profile to distinguish this profile from a different profile (such as a LUT profile made using the same ti3 file but using the colprof "-ax" option). You might also want to add the date the profile was made or whatever other information you might find useful.
- The " \" just means the command continues on the next line; it's not necessary to split the command over two lines, but hopefully this makes it easier for you to read the command without having to scroll across the code box.
- -D"Pentax_K5-II_Sunlight.icc" : a description of the final profile. I usually make the description match or at least start with the profile file name, for use with image editing software that provides profile descriptions but doesn't show the actual file name.
- Pentax_K5-II_Sunlight-cropped-addbw : the base name for the input ti3 file created by scanin, in this case "Pentax_K5-II_Sunlight-cropped-addbw.ti3".
For more information about colprof command line options, see the colprof documentation.
Use xicclu to verify that the resulting camera input profile is also a well-behaved working space profile
My article What Makes a Color Space Well Behaved? gives the criteria for a well behaved RGB working space profile. In a nutshell: if R=G=B, the resulting color is neutral gray; if R=G=B=0, the resulting color is solid black; and if R=G=B=the maximum possible value for the bit depth of the image (65535 for a 16-bit image, 100 for RGB values that have been scaled from 0 to 100), the resulting color is solid white. As long as the Red, Green, and Blue profile tone reproduction curves ("TRCs") are the same (all three TRCs are gamma=1.0 for profiles made using "-am" colprof option), a quick way to tell whether an ICC profile is well behaved is to use xicclu to ask for the RGB values that correspond to neutral LAB values for several points up and down the LAB gray axis. If the profile really is well-behaved, for every LAB value for which LAB_A=LAB_B=0.0, the corresponding RGB values will always have R=G=B.
The ArgyllCMS xicclu utility can be used to to determine what camera input profile RGB values correspond to the LAB values (100,0,0), (50,0,0, (0,0,0) and so forth. The code box below shows the xicclu command line (highlighted in green), the LAB values input (highlighted in yellow — you type in LAB values and xicclu outputs the corresponding camera input profile RGB values), and the resulting xicclu output (not highlighted):
$ xicclu -ir -pl -fif -s100 Pentax_K5-II_Sunlight.icc 100 0 0 100.000000 0.000000 0.000000 [Lab] -> MatrixFwd -> 100.000000 100.000000 100.000000 [RGB] 50 0 0 50.000000 0.000000 0.000000 [Lab] -> MatrixFwd -> 18.418652 18.418652 18.418652 [RGB] 0 0 0 0.000000 0.000000 0.000000 [Lab] -> MatrixFwd -> 0.000000 0.000000 0.000000 [RGB]
Here's what the various xicclu command line options mean:
- -ir : use relative colorimetric conversion
- -pl : use the LAB PCS ("Profile Connection Space")
- -fif : convert from LAB to RGB (rather than the other way around).
- -s100 : scale the LAB and RGB values from 0 to 100.
- Pentax_K5-II_Sunlight.icc : the camera input profile file name
For more information about the xicclu command line options, see the xicclu documentation.
The xicclu terminal output shows that in addition to being a good general purpose camera input profile, the Pentax_K5-II_Sunlight.icc profile is also a well-behaved RGB working space profile and can be used for editing Pentax K5-II interpolated raw files. One situation where this is very useful is if you want to recover and use channel information that would otherwise be clipped upon conversion to any of the standard RGB working space profiles. You might prefer the results of adding saturation in your camera-specific RGB working space, compared to adding saturation in some other RGB working space. Modifying the image white balance in post is best done in your camera-specific RGB working space.
There are many other situations when you might want to edit interpolated raw files in your camera-specific RGB working space, but I'll leave it up to you to experiment. Just be aware that camera-specific RGB working spaces are very large and have a high percentage of their color gamut devoted to entirely imaginary colors (even more than ProPhotoRGB). So use a little caution when adding saturation and make full use of soft proofing techniques when converting to smaller RGB working spaces or to a printer profile.
Summary and Notes
The usual approach to making a camera input profile produces a profile that is highly dependent on the color balance and exposure compensation that was used during raw processing. The resulting profile is very likely to, in fact almost certainly will, add a color cast and it might also add positive or negative exposure compensation (depending on the type of profile you make) to any interpolated raw file to which it is applied.
This article shows how to use ArgyllCMS to make a general purpose custom camera input profile that is:
- Color balanced, so it won't add a color cast to your interpolated raw files.
- Normalized, so it won't make your interpolated raw files artifically darker or lighter than they really are.
- Well-behaved, so it can be used as an RGB working space for editing your interpolated raw files.
Everything you need to make a camera input profile using ArgyllCMS:
- ArgyllCMS: The source code and the latest precompiled executables for Linux, Windows, and Apple are availabled directly from the ArgyllCMS website. ArgyllCMS is almost certainly also available through your Linux repository.
- An IT8 or other camera profiling target. The ArgyllCMS website gives a nice summary of the ArgyllCMS supported target charts. Targets for camera profiling has a nice "cost-benefit analysis" for several different commercially available targets. Personally I use and am very happy with Wolf Faust's IT8 target charts (order the "C1" target chart for profiling cameras; the others are for profiling scanners).
- The reference file that gives the CIEXYZ and/or CIELAB values for your target chart color patches. The reference file is usually supplied by the target chart manufacturer. Or if you have the proper measuring instrument, you can measure the color patches yourself.
- The ArgyllCMS chart recognition template that corresponds to your target chart. If you download ArgyllCMS from the ArgyllCMS website, the chart recognition templates (extension "cht") are located in the ArgyllCMS "ref" folder.
- A properly made target shot, that is, a photograph of your target chart, shot raw and (obviously) taken with the camera that you want to profile. How to make a proper target shot is beyond the scope of this article.
- A raw processor for interpolating the target chart raw file. The raw processor must be capable of outputting a 16-bit linear gamma raw color tiff that hasn't been processed in any way to enhance color or contrast, other than adjusting the color balance and possibly the exposure compensation.
If target charts were perfect . . .
- The target chart would have a range of perfectly neutral color patches, reflecting all wavelengths of light equally.
- The lightest neutral color patch would be the perfect Lambertian reflector, solid white, equally reflecting in all directions all the light illuminating the target, and profiling software could always use the perfect Lambertian reflector color patch to exactly and precisely set the profile white point.
- The darkest neutral color patch would be the perfect black trap, solid black, reflecting no light at all.
- Every single color patch on the target chart, including of course all the neutral color patches, would be perfectly characterised by the corresponding XYZ values in the provided reference file.
- The target chart would include a full range of real colors covering the entire "Lab Gamut. . . . the set of Lab color coordinates for which there could possibly be a physical sample. These are the "real colors."
- Illuminant metameric failure would never be a problem with the target chart color patches.
In the real world of affordable target charts:
- The neutral color patches on the target chart aren't completely neutral.
- The lightest neutral patch (which isn't really neutral, but rather "paper white") will only reflect around 80 to 90 percent of the ambient light.
- The darkest neutral patch (which also isn't really neutral, but rather only as neutral as the pigments that were used to print the darkest neutral patch) still reflects around 5 to 10 percent of the ambient light.
- There are limits to the accuracy with which the color patches can be measured, so there are unavoidable small errors in the target chart reference file, and more so if the target chart is batch-measured rather than individually measured.
- Many real colors can't be printed, and cost considerations further limit the range of colors on any given target chart. So the target chart is quite limited as to the range of real colors that are represented.
- Metameric failure (the camera doesn't "see" what a human would see) does affect target chart color patches.
Considering all the limitations of ideal vs real target charts, the ArgyllCMS profiling software can't rely 100% on the accuracy of the XYZ refererence values for any of the color patches, including the neutral patches. Also, the ArgyllCMS profiling software has no way to second-guess what white balance was used to interpolate the raw target shot, or what the in-camera exposure was, or what raw exposure compensation might have been used while processing the target shot. Rather, it has to take the data you give it at face value.
Links to pertinent discussions from the ArgyllCMS mailing list archives:
The ArgyllCMS mailing list searchable archives contain much good information about making a proper target shot and about profiling in general. Here are some pertinent threads (the information in these threads covers much more territory than the rather specific post topics would indicate):