Finding an Image Inside Another Image

Question

I'm trying to build an application that solves a puzzle (trying to develop a graph algorithm), and I don't want to enter sample input by hand all the time.

Edit: I'm not trying to build a game. I'm trying to build an agent which plays the game "SpellSeeker"

Say I have an image (see attachment) on the screen with numbers in it, and I know the locations of the boxes, and I have the exact images for these numbers. What I want to do is simply tell which image (number) is on the corresponding box.

So I guess I need to implement

bool isImageInsideImage(Bitmap numberImage,Bitmap Portion_Of_ScreenCap) or something like that.

What I've tried is (using AForge libraries)

public static bool Contains(this Bitmap template, Bitmap bmp)
{
    const Int32 divisor = 4;
    const Int32 epsilon = 10;

    ExhaustiveTemplateMatching etm = new ExhaustiveTemplateMatching(0.9f);

    TemplateMatch[] tm = etm.ProcessImage(
        new ResizeNearestNeighbor(template.Width / divisor, template.Height / divisor).Apply(template),
        new ResizeNearestNeighbor(bmp.Width / divisor, bmp.Height / divisor).Apply(bmp)
        );

    if (tm.Length == 1)
    {
        Rectangle tempRect = tm[0].Rectangle;

        if (Math.Abs(bmp.Width / divisor - tempRect.Width) < epsilon
            &&
            Math.Abs(bmp.Height / divisor - tempRect.Height) < epsilon)
        {
            return true;
        }
    }

    return false;
}

But it returns false when searching for a black dot in this image.

How can I implement this?

Answer 1

I'm answering my question since I've found the solution:

this worked out for me:

System.Drawing.Bitmap sourceImage = (Bitmap)Bitmap.FromFile(@"C:\SavedBMPs\1.jpg");
            System.Drawing.Bitmap template = (Bitmap)Bitmap.FromFile(@"C:\SavedBMPs\2.jpg");
            // create template matching algorithm's instance
            // (set similarity threshold to 92.5%)

           ExhaustiveTemplateMatching tm = new ExhaustiveTemplateMatching(0.921f);
                // find all matchings with specified above similarity

                TemplateMatch[] matchings = tm.ProcessImage(sourceImage, template);
                // highlight found matchings

           BitmapData data = sourceImage.LockBits(
                new Rectangle(0, 0, sourceImage.Width, sourceImage.Height),
                ImageLockMode.ReadWrite, sourceImage.PixelFormat);
            foreach (TemplateMatch m in matchings)
            {

                    Drawing.Rectangle(data, m.Rectangle, Color.White);

                MessageBox.Show(m.Rectangle.Location.ToString());
                // do something else with matching
            }
            sourceImage.UnlockBits(data);

The only problem was it was finding all (58) boxes for said game. But changing the value 0.921f to 0.98 made it perfect, i.e. it finds only the specified number's image (template)

Edit: I actually have to enter different similarity thresholds for different pictures. I found the optimized values by trying, in the end I have a function like

float getSimilarityThreshold(int number)