In this project we explore the use of image quilting for texture synthesis and transfer. Texture synthesis involves generating a larger texture image by stitching together 'similar' patches from a source image while texture transfer involves taking an input image and giving it the texture of a source image. The methods used are based on those from Efros et. al.
Reveal Source Textures
We can create a poor-quality quilt by going in raster scanning order over the output image and placing randomly chosen patches from the input image. This lazy method makes no effort to create a congruous result, but serves as a baseline to beat.
Starting by randomly sampling the initial patch, we choose the preceding patches by choosing a patch that is sufficiently similar, by measure of the sum of square differences (SSD), between all possible patches and the overlapping region the new patch will be placed in.
We can improve the quality of the quilt by finding the seams between patches. We can do this by finding the minimum cost path along the overlapping region of the new patch. Here, the cost is the SSD between pixel values of the overlapping and existing texture (over the 3 color channels).
To the right you can see two overlapping regions of two randomly chosen patches along with the corresponding cost-map with the shortest path shown as points.
To implement texture transfer, we introduce another cost term based on the difference between the sampled patch and the corresponding location on the target image to be textured. Essentially, we would like to combine the low frequencies of the target image with the sharpness of the sampled texture.