Leaves in May — A tutorial on making an illustration from a photograph
This tutorial explains a method for making an illustration rendering from a photograph. One of the steps produces a nice line drawing. So really this is two tutorials in one.
Written December 2016.
This tutorial explains a method for making an illustration rendering from a photograph, using my Leaves in May image to show the individual steps. One of the steps produces a nice line drawing. So really this is two tutorials in one. Figure 1 shows the major steps along the way from the scene-referred interpolated camera raw file to the final Leaves in May illustration.
Here's an overview of the procedure I've been experimenting with to make illustrations from photographs:
- First, working in a linear gamma RGB color space, convert the image to black and white and establish the initial image tonality (Figure 1, Slide 2), preferably starting from a scene-referred rendering of a camera raw file (Figure 1, Slide 1).
- Second, make a simplified version of the image from step 1.
- Third, working in a perceptually uniform RGB color space, make copy of the simplified image from step 2 and turn the copy into a line drawing (Figure 1, Slide 3).
- Fourth, still working in a perceptually uniform RGB color space, multiply the simplified image by the line drawing to make the initial illustration (Figure 1, Slide 4).
- Fifth, convert the resulting initial illustration from step 4 to a linear gamma RGB working space. Then add color and modify the tonality to suit (Figure 1, Slide 5).
Establish initial tonality
Preferably start with a scene-referred rendering of a camera raw file
What is a scene-referred rendering?
In a scene-referred rendering of a camera raw file, to the extent possible the RGB channel values in the interpolated file are proportional to the intensities of the light that entered the camera as recorded by the sensor:
- Operations that preserve proportionality include exposure compensation, white balancing, and multiplying or dividing the entire image by a solid color.
One also can put "repair" operations such as noise removal, highlight restoration, black frame subtraction, and correcting lens distortions and vignetting into this category, as the scene that was photographed certainly did not suffer from noise, blown highlights, lens distortions/vignetting and so forth.
- Operations that modify proportionality (and thus produce output that is not scene-referred) include making most Curves adjustments (including darktable-style camera curves), increasing or decreasing saturation/vibrancy/etc, and just about any operation that you might do in the LAB or LCH color space.
Why is it usually preferable to start with a scene-referred rendering?
Of course it depends on your artistic intentions for the image. But as a general rule of thumb, the kinds of tonality and color changes that are suitable for normal photographic rendering aren't suitable as a starting point for "not-so-normal" rendering such as making an illustration from a photograph, emulating various Pictorialist styles, and such, for at least the following two reasons:
- First, the closer the image is to a scene-referred rendition, the more information there is to work with when making non-standard renderings of photographs.
- Second, our perception of contrast in an image changes fairly radically by the addition or removal of color information. As (1)the next step in making an illustration from a photograph is to make a conversion to black and white, and (2)color won't be added back to the image until much later in the processing, there's not a lot of point in working to establish a finished "normal" photographic rendering.
Figure 2 below shows the GIMP-CCE layer stack I used to establish the initial tonality for Leaves in May. The initial processing was done in a linear gamma version of the Rec.2020 color space (Rec2020-elle-V4-g10.icc from my github ICC profile pack). The scene-referred interpolated raw file is at the bottom of the stack:
Convert to black and white
To convert the image to black and white, make a copy of the interpolated raw file layer and make a relative luminance conversion to black and white:
There are a million ways to convert an image to black and white. The simplest, and for "alternative" photographic processing, usually the best, is to make a straight relative luminance conversion:
- Most raw processors probably offer a straight "relative luminance" conversion to black and white. I'll add the information here if anyone can supply the right commands for the various raw processors.
- In GIMP-CCE, to make a Luminance conversion to black and white, choose "Colors/Desaturate/Luminance". Your image should be in your preferred linear gamma RGB working space. Otherwise you won't get a relative luminance conversion to black and white; instead you'll get the equivalent of the default GIMP 2.9 "Luma" conversion to black and white.
- In default GIMP-2.9, convert your image to the built-in sRGB color space, and choose "Colors/Desaturate/Desaturate/Luminance" — that's not a typo — "Colors/Desaturate" pulls up a second menu, with one entry being "Desaturate".
Make initial tonality adjustments
There are a million ways to adjust tonality, depending entirely on your artistic intent. But making an illustration rendering of a photograph involves fairly drastic departures from normal "straight photographic" rendering. So it's best to confine the initial tonality adjustments to two adjustments, both of which should be done in a linear gamma RGB working space:
- If the image has a lot of detail that's lost in the shadows, pull out the shadow information. Avoid using tone-mapping algorithms that severely distort the image tonality. This step isn't intended to produce anything near the final image tonality. It's just a preliminary processing step so you can see what detail you have to work with.
- Establish a dynamic range that isn't too extreme, meaning the darkest dark areas shouldn't be solid black and the lightest light areas shouldn't be solid white — you want a little room for subsequent tonality modifications. I'd recommend setting the image black and white points such that the majority of the image has Lightness values higher than around 5 and lower than around 95. For some images and alternative processing techniques, a Lightness range of Lightness=10 to Lightness=90 works even better.
For Leaves in May, for pulling out the shadow detail I followed the procedure described in Tone mapping and shadow recovery using GIMP's 'Colors/Exposure'. For this image I didn't need to make any additional tonal modifications to set the dynamic range. But if I had needed to adjust the dynamic range, I would have used Levels to set the black and white points, using GIMP's LCH color picker to monitor the Lightness values in the shadows and highlights. In default GIMP, use "Colors/Exposure" because default GIMP Levels and Curves so far (as of September 2017) do not linearize the sRGB TRC.
Putting it all together, Figure 5 shows the GIMP-CCE layer stack that I used to establish the initial tonality for Leaves in May.
Make the initial illustration rendering
Converting to black and white and establishing the initial tonality (Section B above) is best done in a linear gamma RGB working space. Making the initial illustration rendering (this Section) is best done in a perceptually uniform RGB working space.
Starting from the already established initial tonality, the GIMP-CCE layer stack for making the initial illustration rendering for Leaves in May is shown in Figure 6. The bottom layer labeled "4 New from Visible (labl TRC)" was produced by dragging the top layer from Figure 5 out as a new image and converting the new image from the linear gamma "Rec2020-elle-V4-g10.icc" color space to the perceptually uniform "Rec2020-elle-V4-labl.icc" color space:
Simplify the details (Layer 5)
There are a million ways to simplify the details in an image, including:
- Down-sizing the image and then up-sizing back to the original image size. Try using a linear gamma color space to down-size the image and a perceptually uniform color space to up-size. Or start with a crop from a larger image, and up-size it.
- Noise-removal algorithms: use stronger parameters than you normally would because softening of details is exactly what you are after.
- Blurring algorithms: Krita provides an excellent lens blur algorithm, which should be done in a linear gamma color space, as should Gaussian blur.
Technically speaking probably all blurring algorithms should be done in a linear gamma color space, and most of the time blurring in a linear gamma color space does also produce nicer results. But the world won't end if you choose to blur an image in a perceptually uniform color space.
- Wavelets: remove one or two of the detail layers.
- G'MIC supplies a huge number of suitable algorithms under "Degradations" and "Repair".
Please note that G'MIC expects to receive images that are in the regular sRGB color space. On the one hand, for many G'MIC operations, all RGB working spaces will produce technically correct results as long as you stick with processing the RGBA channels. On the other hand, if you like the results, whether the operation was technically correctly done or not is perhaps a secondary concern.
I've experimented with various (by no means all possible!) simplifying algorithms, and depending on what you want to accomplish, some algorithms will work better than others. To simplify the Leaves in May details I did a 2-pixel median blur after converting the image from a linear gamma color space to a perceptually uniform color space. But depending on the algorithm and your artistic intentions you might prefer results from working in a linear gamma RGB working space to simplify the image.
Find edges (Layer 6)
The next step is to make a copy of the simplified image and run a "find edges" algorithm. After experimenting quite a bit, I've concluded that "Find edges" algorithms produce more useable results when done in perceptually uniform RGB color spaces.
For Leaves in May, to find the edges I used GIMP's "Prewitt-compass" algorithm, using an Amount of 1.000 and setting the Border behavior to "Clamp":
I like using algorithms that make finer edges, but if your style demands more pronounced edges, of course choose the algorithm and radius accordingly. I haven't yet experimented with using G'MIC's edges ("Contours") algorithms in an actual workflow, but I've looked at them and they look very promising.
Optional step: Dilate or Erode
There are two other algorithms that you might want to use on the simplified layer before running the "Find Edges" algorithm, namely "Erode" or "Dilate". In default GIMP, Dilate and Erode are found under "Filters/Generic". In GIMP-CCE use "Filters/Generic/Value Propagate".
Dilate grows lighter areas of the image and Erode grows darker areas of the image. In the present context, the purpose of using Dilate or Erode before running the Find Edges algorithm is to create "slightly relocated edges" that don't extend past the actual edges of the main objects in the original image. This works really well, but I'll leave it to you to experiment. If your image is predominately lighter objects against a dark background, try Erode, and if your image is predominantly darker objects against a light background, try Dilate.
Invert the result of "Find Edges" (Layer 7)
After running the "Find Edges" algorithm, the next step is to invert the colors in the "Edges" layer (well, it's a black and white image, but gray is also a color!). This can be done either directly on the "Edges" layer or on a copy:
- You might try running a Levels Gamma adjustment before or after inverting the colors. For Leaves in May I made a Gamma=1.9 adjustment before inverting the colors (see Figure 7 above).
- For a radiometrically correct result, color inversion should be done in a linear gamma RGB color space. But for the current purpose (making illustration renderings and also when making line drawings), usually I prefer the results from making the color inversion on perceptually uniform RGB.
Results from inverting colors and also from multiplying in linear gamma vs perceptually uniform color spaces can be fairly drastically different, so it's worth doing some experimenting to see what you like, and you might prefer one or the other depending on the image and your artistic intentions.
In GIMP-CCE experimenting with inverting colors and with multiplying layers together is easily done by converting the image back and forth between an RGB color space with a perceptually uniform TRC and the same color space with a linear gamma TRC. In default GIMP use "Colors/Linear Invert" for inverting linearized sRGB, and "Colors/Invert" for inverted using the regular sRGB TRC, which is nearly perceptually uniform.
This is what Layer 8 looks like before you change the blend mode from Normal to Multiply. At this point you have a pretty nice "line drawing from a photograph", and if you wanted to, you could paint in colors to make a color line drawing. If you stop right here, you have a nice line drawing made from a photograph. The steps to continue on to make an illustration-style rendering from a photograph are described below.
Set the "inverted edges" layer (Layer 8) to Multiply blend mode over the "simplified" layer (Layer 5), and make a new layer from the result (Layer 9)
To make the initial illustration from the accumulated layers:
- Hide the intermediate layers so that only the inverted layer (Layer 8) and the simplified layer (Layer 5) are visible (or else the "simplified+dilated/eroded" layer, depending on whether you applied a dilate or erode algorithm to the simplified layer).
- Then set the inverted layer to Multiply blend mode. (If you are using default GIMP 2.9, things are a bit more complicated and you'll need to use "Perceptual gamma precision" to get the Multiply blend mode to work on perceptually uniform RGB.)
- Then make a new layer from the result of multiplying the "simplified" layer by the "inverted edges" layer (whether in GIMP-CCE or default GIMP, the command is "Layer/New from Visible"). This new layer (shown in Figure 9 below) is the initial illustration rendering and will be converted back to a linear gamma RGB working space for modifying the tonality and adding some color.
Add color and/or modify the tonality to suit
The initial tonality for Leaves in May was established in a linear gamma RGB color space (Section B above). The initial illustration rendering was done in a perceptually uniform RGB color space (Section C above). The next (and last) task is to establish the final color and tonality, for which it's best to convert the image back to a linear gamma RGB working space.
Some color mixing and color appearance considerations
Radiometrically correct and aesthetically pleasing color mixing:
For various reasons modifying tonality and adding or modifying color only produces radiometrically correct results — and also usually produces more aesthetically pleasing results (at least in my opinion, your opinion might be different!) — when working in a linear gamma RGB color space. So the next step is to drag the initial illustration rendering (which is Layer 9 from the layer stack for Section C) out as a new image, and convert the new image to a linear gamma RGB working space.
For Leaves in May, I converted the image back to Rec2020-elle-V4-g10.icc. Things are more complicated in default GIMP and I don't feel like explaining the complications and workarounds in this tutorial. So the rest of this tutorial doesn't cover using default GIMP 2.9.
Color appearance: modifying color affects perceived tonality and contrast
I've noticed that changing the colors in an image has an immediate effect on the image's perceived tonality and contrast, even if the image's Luminance/Lightness is kept constant. This is a topic that Color Appearance Models address. For example, more colorful colors also look brighter than less colorful colors, even when the Luminance/Lightness is held constant.
This inconvenient fact that "color modifies the perceived tonality and contrast" (and vice versa) is not accomodated by normal RGB/LAB/LCH image editing (the LAB/LCH color spaces model color differences, but not color appearances).
The practical implication from all of the above is that to keep color changes and tonality changes working together to produce your final desired image, in today's digital darkroom you just have to rely on your aesthetic sense to adjust the image colors as you change the tonality, and vice versa. So even though below I show the tonality modifications separately from the color modifications, in reality I always end up working back and forth between modifying tonality and modifying color. I find that this "back and forth" process is made much easier by editing color and tonality separately.
Modify the tonality and add color to suit
Figure 10 below shows the GIMP-CCE layer stack for modifying the initial illustration produced in Section C above to make the final Leaves in May image. The layer stack has a separate Layer Group for modifying tonality (see Figure 11 for the final tonality). And there are two additional layers above the tonality group for adding color to what is so far a black and white image (see Figure 12 for the final image with the color added).
In Figure 10, you can see that all the tonality adjustments are in the Layer Group labeled "Tonality, Normal blend". Most of the layers in the Tonality Layer Group use Normal blend mode. But three layers use the Soft light blend mode, and these layers are labeled to indicate the blend mode.
For the most part there's no need to describe the purpose of the individual tonality adjustment layers beyond the information shown in the layer labels. However, the layers labeled "12=Mask from 11, Soft light to flatten tonality" and "13=Visible, Soft light" might be worth noting. I had two goals for modifying the image resulting from the initial illustration rendering from Section C above:
- Make the image brighter (layers 10, 11, and 14).
- Decrease unwanted local contrast to reduce the sense of grittiness and restore a sense of "leaves and light" (layers 12 and 13).
Often Soft light blend is used to gently increase local contrast. But when done in a linear gamma color space using the inverse of a grayscale mask (as per layer 12 in Figure 10 below), the net effect is to flatten local contrast. So layer 12 uses the Soft light blend to significantly flatten local contrast, and layer 13 uses the Soft light blend to restore contrast to just the desired areas.
You might not be familiar with using GIMP's LCH blend modes to add color to a black and white image. In the layer stack shown in Figure 10 above, the top two layers (just below the Final image layer) use the LCH Chroma and Color blend modes, respectively, and these two layers turn the black and white image into a monochrome color image. In the resulting color image:
- The LCH Hue is constant at 117 (a warm spring green).
- The LCH Chroma (roughly "how saturated") runs from Chroma=9 in the shadows to Chroma=5 in the highlights.
I added a Chroma layer so that I could make the shadows have a slightly higher Chroma than the highlights (which is exactly the opposite of what you'd normally do, for reasons that are too complicated to explain in this tutorial). The layer labels show the actual LCH values for each of the two LCH layers, and Figure 12 below shows the result of adding color to the image.
On the deceptive linearity of a tutorial
Sometimes (rarely) I'll look at a "from the camera" image and know exactly how I want the final image to look and what editing steps I'll use to get there. Leave in May was not one of these images. I stared at the thumbnail embedded in the raw file for years before deciding I really did want to work on the image. Producing the initial illustration rendering was easy enough because I'd been experimenting with the procedure on several other images. But getting from the initial illustration rendering to the final image was a struggle. If you could see the pile of discarded XCF files I produced on my way from "start" to "finish" you'd probably laugh. So don't let the linearity of this tutorial fool you into thinking that the actual making of the image involved a clear plan for getting from the raw file to the finished image.
Notes on taking the photograph from which the Leaves in May illustration was made
I think whatever charm or appeal my Leaves in May image might have is from the lens that was used to capture the original scene, well, that and of course in my good taste (or more likely the fortuitous accident) in where the camera was pointing when I pushed the shutter.
My camera at the time was the Canon 400D, a good entry level camera but certainly nothing special. But the lens really is a bit special. One of the nice things about Canon cameras (and also about the new mirrorless cameras) is that they can be fitted with many different lenses made by many different lens manufacturers. The initial capture for the Leaves in May image was made using an old manual focus micro-Nikkor 55mm f2.8 lens, which produces exceptionally nice bokeh for out-of-focus areas. All these years later (and now using the Sony A7 mirrorless camera), this venerable old lens is still my absolute favorite lens.
Notes on making line drawings and illustrations in different RGB working spaces
Many RGB editing operations will produce different results in different RGB working spaces (sometimes radically different), depending on the RGB color space primaries and the particular editing operation. But once an image is converted to black and white, the color space primaries of your chosen RGB working space no longer matter as all the colors are on the gray axis and the gray axis is the same in all ICC RGB working color spaces.
Also a luminance conversion to black and white (which is done as part of the preliminary processing for making a line drawing or illustration from a photograph) is completely independent of the RGB working space in which the conversion is done (unless of course you clipped colors by converting from your camera input profile to your RGB working space at integer precision instead of floating point precision). Many ways of converting to black and white are very dependent on the color space primaries. But relative luminance is a property of the actual image colors as located in the XYZ color space.
So although I chose to work in linear gamma and perceptually uniform versions of the Rec.2020 color space to make Leaves in May, the specific procedures that I use for making an illustration from a photograph can actually be done in any RGB working color space and still produce the exact same results, assuming of course that you have linear gamma and perceptually uniform versions of your preferred RGB working space.
However, please note that the precise TRC ("Tone Reproduction Curve") for the perceptually uniform version of your preferred RGB working space does make a different for the steps that are done using perceptually uniform RGB. Color spaces with the labl TRC, the sRGB TRC, and the gamma=2.2 TRC will all produce slightly different results even when editing a black and white image. For a simple example, making a step wedge from a black to white gradient produces different results depending on the RGB color space TRC:
Once color is thrown into the mix (or back into the mix as in the current case), of course the RGB color space primaries do matter quite a lot for many editing operations, for example changing the white balance or using Multiply to add a color cast.
But for Leaves in May, the only color operations use the LCH color space, and for these operations the RGB color space primaries don't matter. Why? The first step in the LCH algorithms is to convert the image from RGB to XYZ, and then to LAB and LCH. And processing the image in any of these reference color spaces produces results that are independent of the RGB working space (assuming your image editor isn't hard-coded to a given RGB color space, as default GIMP 2.9 is hard-coded to use sRGB, but please note that "future" default GIMP will support editing in user-chosen RGB working spaces).
Other articles that I've written in my on-going exploration of combining painting and photography
Well, hopefully you enjoyed this tutorial, and if you have suggestions for improving it, please share! If you did enjoy this tutorial, you might also like the following articles that show some of the ways I've been exploring for combining painting and photography:
- Combining painting with photography. Much of my inspiration for wanting to combine painting and photography comes from studying Pictorialist photographs from the late 19th/early 20th century. This article illustrates several approaches that I've been exploring as ways to incorporate painting with photography.
- Painting and blending colors using Addition, Subtract, Divide, and Multiply blend modes. This article uses digital "glazing grids" to demonstrate the various ways RGB colors can be mixed in the digital darkroom or painting studio, and then shows the layer stacks for two paintings done using Addition, Divide, and Multiply layer and brush blend modes.
- Urn in a Window: A painting in progress, using a workflow that can also be followed when colorizing or split-toning black and white photographs