How does Content-Aware fill work?

algorithm graphics image-processing artificial-intelligence photoshop
CiscoIPPhone picture CiscoIPPhone · Mar 27, 2010 · Viewed 17.9k times · Source

In the upcoming version of Photoshop there is a feature called Content-Aware fill.

This feature will fill a selection of an image based on the surrounding image - to the point it can generate bushes and clouds while being seamless with the surrounding image.

See http://www.youtube.com/watch?v=NH0aEp1oDOI for a preview of the Photoshop feature I'm talking about.

My question is: How does this feature work algorithmically?

Answer

danbgoldman picture danbgoldman · Apr 22, 2014

I am a co-author of the PatchMatch paper previously mentioned here, and I led the development of the original Content-Aware Fill feature in Photoshop, along with Ivan Cavero Belaunde and Eli Shechtman in the Creative Technologies Lab, and Jeff Chien on the Photoshop team.

Photoshop's Content-Aware Fill uses a highly optimized, multithreaded variation of the algorithm described in the PatchMatch paper, and an older method called "SpaceTime Video Completion." Both papers are cited on the following technology page for this feature:

http://www.adobe.com/technology/projects/content-aware-fill.html

You can find out more about us on the Adobe Research web pages.

Related questions

What is the idea behind scaling an image using Lanczos?

I'm interested in image scaling algorithms and have implemented the bilinear and bicubic methods. However, I have heard of the Lanczos and other more sophisticated methods for even higher quality image scaling, and I am very curious how they work. …

Read More
efficient algorithm for drawing circle arcs?

I am using the mid-point circle algorithm (bresenham circle) to efficiently draw whole circles. Is there something similar to draw circle arcs? I would like to specify a start angle and end angle and have only that portion of the …

Read More
Algorithm to detect intersection of two rectangles?

I'm looking for an algorithm to detect if two rectangles intersect (one at an arbitrary angle, the other with only vertical/horizontal lines). Testing if a corner of one is in the other ALMOST works. It fails if the rectangles …

Read More