蚕吐丝的过程:Project III -?Gradient-Domain Fusion

CS 498 Computational Photography
Project III - Gradient-Domain Fusion
Jia-Bin Huang
Website:https://netfiles.uiuc.edu/jbhuang1/www/
Email: jbhuang1[at]illinois.edu
1. Introduction
The goal of this project is to seamlessly fuse two images using gradient domain processing. The main idea of gradient-domain image fusion is that human visual system does not perceive absolute pixel values, but instead relies upon local contrast and ratios (i.e., gradients in the images). Gradient-domain processing can directly manipulate pixel differences in addition to the pixel values in the images, thus result in many usful image filtering, editing applications such as image blending, saliency sharpening, color2gray, de-blocking, sparse data interpolation (e.g., colorization of a gray image), and non-photorealistic rendering. In this project, we start with Poisson image blending and then design and implement various filters (saliency sharpening, color2gray, and non-photorealistic filter).
2. Graphical User Interface
We created a Graphical User Interface (see Figure. 1 below) for users to select images and test out diferent gradient-domain processing applications. Five applications using gradient-domain processing methods have been implement: 1) Poisson image blending, 2) Poisson image blending with mixed gradient, 3) saliency sharpening, 4) non-photorealistic rendering filter, and 5) color2gray. The user can use the slider bar in the bottom right to control the parameters used in different applications to get the desired results (e.g., the degree of enhancement).

Figure. 1 Sreenshot of the graphical user interface.
3. Gradient Domain Fusion
3.1 Poisson Image Blending
We use the gradient domain optimization framework for Poisson image blending. With user-specified mask, the data constraint sure the pixel value consistancy along the mask edges. On the other hand, the gradient constraints enforce the details in the source image. Figure 2, we show the use of Poisson image blending for face replacement. The source and the target images are the Afghan girls in 1984 and 2001, respectively. Here we cropped the younger face from the source image and paste it onto the portrait 17 years later. We can see that in (c) the naive copy and paste approach cannot work since the colors are quite different, causing significant artifacts. With Poisson image blending, we can fix up the discontinuty along the boundary and match the colors. The resultant image is the older Afghan girl with a younger face (17 years ago).

(a) Source image (b) Target image (c) Composite image using copy and paste (d) Composite using Poisson blending
Figure 2. Poisson image blending result.
Figure 3 shows another example of Poisson image blending. The selected object is seamlessly inserted to the target image. However, there is is intensity difference between the target image and the selected object. Therefore, the overall intensity of the object is decreased. Some of the image details have lost during the process. This is known disadvantage of using Poisson image blending.



(a) Source image
(b) Target image
(c) Composite using Poisson blending
Figure 3. Example of Poisson blending.
Figure 4 shows the failure case of Poisson image blending. Here the selected object in (a) contains very high frequency details in the selected region. Also, the target image contain quite different textures. Thus, the optimization criteria are hard to satify to get a seamless boundary. In this case, the selected object boundary can be perceived in the composited image.

(a) Source image
(b) Target image
(c) Composite using Poisson blending
Figure 4. Failure case of Poisson blending.
3.2 Blending with Mixed Gradients
The original formulation of the Poisson image blending method ignore all the gradients in the target image. Thus, when the source image contains large smooth area, the blending results will look blurred and unnatural. Take the Figure 6 below as an example. The source image is the blackboard of renowned physicist, Prof. Richard Feynman. The target image is a brick wall. Using original formulation of the Poisson image blending will result in image (c). We can see that the textures in the bottom left has been wiped lot (because we completely ignore gradient information in the target image). On the other hand, the blending with mixed gradient generates much better result, see image (d). The texture of the brick wall from the target image and the scribbles in the source image can both be well preserved.


(a) Source Image
(b) Target Image

(c) Normal Poisson blending
(d) Blending with mixed gradients
Figure 5. Example result of Poisson image blending with mixed gradient.
4. Bells and Whistles
4.1. Saliency Sharpening
We use the gradient-domain optimization framework to design a saliency enhancement filter. To enhance the salient regions in the image, we enhance the gradients along the edges. There are thus two parameters in designing the saliency sharpening filter. The first one is the degree of enhancement, which controls how much gain of the image gradient we want to increase. The second parameter, edge enhancement, further enhance the gradients for edge pixels. We specify the data constraint as the original image and the gradient constraints as the enhanced gradient field.






(a) Original image
(b) Saliency sharpening filter results
Figure 6. Sample results of saliency sharpening filtering.
4.2. Color2Gray
When projecting color image to gray scale images, all the information in color channels are lost. Thus, some important image content might not be perceived in the gray-scale image. To preserve the color information, we modified the gradient field so that the color contrast can be also be well preserved. We convert the color image to the CIELab color space where the Euclidean distance approaximately corresponds to the contrast perception in human vision. In addition to the original gradient contrast in the luminance channel, we add the gradient contrast into the intensity contrast. However, the color contrast computed in the CIELab color space is not directional (i.e., without positive or negative sign). We follow the Color2Gray paper and introduce a parameter theta that specify a plane to determine the positive or negative signs of the color contrast. Figure 7 shows the sample results of using Color2Gray algorithm and compare with the grayscale conversion using luminance channel.












(a) Original color image.
(b) Color to gray conversion by perserving the luminance channel only
(c) The results by the Color2Gray methods
Figure 7. Sample results of the Color2Gray algorithm.
4.3 Non-Photorealistic Rendering
We design the non-photorealistic rendering filter by manupilating the gradient field such that the gradients along the edges are amplified while suppressing the gradients in the non-edge regions.








(a) Original color image
(b) Non-photorealistic rendering
Figure 8. Sample results of the non-photorealistic filtering.
5. Download
Here you can download the graphical user interface and play with your own images. Have fun!
Download