How to remove a "green screen" portrait background

Question

I'm looking for a way to automatically remove (=make transparent) a "green screen" portrait background from a lot of pictures.

My own attempts this far have been... ehum... less successful.

I'm looking around for any hints or solutions or papers on the subject. Commercial solutions are just fine, too.

And before you comment and say that it is impossible to do this automatically: no it isn't. There actually exists a company which offers exactly this service, and if I fail to come up with a different solution we're going to use them. The problem is that they guard their algorithm with their lives, and therefore won't sell/license their software. Instead we have to FTP all pictures to them where the processing is done and then we FTP the result back home. (And no, they don't have an underpaid staff hidden away in the Philippines which handles this manually, since we're talking several thousand pictures a day...) However, this approach limits its usefulness for several reasons. So I'd really like a solution where this could be done instantly while being offline from the internet.

EDIT: My "portraits" depictures persons, which do have hair - which is a really tricky part since the green background will bleed into hair. Another tricky part is if it is possible to distingush between the green in the background and the same green in peoples clothes. The company I'm talking about above claims that they can do it by figuring out if the green area are in focus (being sharp vs blurred).

Answer 1

Since you didn't provide any image, I selected one from the web having a chroma key with different shades of green and a significant amount of noise due to JPEG compression.

There is no technology specification so I used Java and Marvin Framework.

input image:

The step 1 simply converts green pixels to transparency. Basically it uses a filtering rule in the HSV color space.

As you mentioned, the hair and some boundary pixels presents colors mixed with green. To reduce this problem, in the step 2, these pixels are filtered and balanced to reduce its green proportion.

before:

after:

Finally, in the step 3, a gradient transparency is applied to all boundary pixels. The result will be even better with high quality images.

final output:

Source code:

import static marvin.MarvinPluginCollection.*;

public class ChromaToTransparency {

    public ChromaToTransparency(){
        MarvinImage image = MarvinImageIO.loadImage("./res/person_chroma.jpg");
        MarvinImage imageOut = new MarvinImage(image.getWidth(), image.getHeight());
        // 1. Convert green to transparency
        greenToTransparency(image, imageOut);
        MarvinImageIO.saveImage(imageOut, "./res/person_chroma_out1.png");
        // 2. Reduce remaining green pixels
        reduceGreen(imageOut);
        MarvinImageIO.saveImage(imageOut, "./res/person_chroma_out2.png");
        // 3. Apply alpha to the boundary
        alphaBoundary(imageOut, 6);
        MarvinImageIO.saveImage(imageOut, "./res/person_chroma_out3.png");

    }

    private void greenToTransparency(MarvinImage imageIn, MarvinImage imageOut){
        for(int y=0; y<imageIn.getHeight(); y++){
            for(int x=0; x<imageIn.getWidth(); x++){

                int color = imageIn.getIntColor(x, y);
                int r = imageIn.getIntComponent0(x, y);
                int g = imageIn.getIntComponent1(x, y);
                int b = imageIn.getIntComponent2(x, y);

                double[] hsv = MarvinColorModelConverter.rgbToHsv(new int[]{color});

                if(hsv[0] >= 60 && hsv[0] <= 130 && hsv[1] >= 0.4 && hsv[2] >= 0.3){
                    imageOut.setIntColor(x, y, 0, 127, 127, 127);
                }
                else{
                    imageOut.setIntColor(x, y, color);
                }

            }
        }
    }

    private void reduceGreen(MarvinImage image){
        for(int y=0; y<image.getHeight(); y++){
            for(int x=0; x<image.getWidth(); x++){
                int r = image.getIntComponent0(x, y);
                int g = image.getIntComponent1(x, y);
                int b = image.getIntComponent2(x, y);
                int color = image.getIntColor(x, y);
                double[] hsv = MarvinColorModelConverter.rgbToHsv(new int[]{color});

                if(hsv[0] >= 60 && hsv[0] <= 130 && hsv[1] >= 0.15 && hsv[2] > 0.15){
                    if((r*b) !=0 && (g*g) / (r*b) >= 1.5){
                        image.setIntColor(x, y, 255, (int)(r*1.4), (int)g, (int)(b*1.4));
                    } else{
                        image.setIntColor(x, y, 255, (int)(r*1.2), g, (int)(b*1.2));
                    }
                }
            }
        }
    }

    public static void main(String[] args) {
        new ChromaToTransparency();
    }
}