How to replace a masked part of an image with another image using openCV python?

Question

The objective is to replace a part of an image with another image. What I plan to do is to get the segment map of the original photo, and replace the selected part of the original with another image.

For example, This is a photo of the original photo's segment map:

And I want to replace that pink segment mask with another guy model (different person):

How should I do so about it? I am thinking of selecting an ROI based on the pink colour (RGB Values of 192,128,128) and bitwise_and the two images? But I'm not too sure on how to do that, or if that's the best approach.

I am aware that the pink mask and the sample guy are not the exact fit, but I just want to be able to fit the model in the pink segment first, and perhaps scale or transform it afterwards.

Any suggestions will be great! Thank you!

Answer 1

Load libs and read images

from __future__ import division
import cv2
import numpy as np

guy_img = cv2.imread('path')
target_img = cv2.imread('path')

Get bounding boxes for the guy (every pixel that is non-zero) and for the target mask (the pink region).

a = np.where(guy_img > 0)
b = np.where(target_img == 129)  # picked one of the channels in your image
bbox_guy = np.min(a[0]), np.max(a[0]), np.min(a[1]), np.max(a[1])
bbox_mask = np.min(b[0]), np.max(b[0]), np.min(b[1]), np.max(b[1])

Note that when I loaded the target image in the values were different from the ones you provided. There were also some white pixels along the edges of the image. That was likely just due to the image being uploaded to imgur and downloaded. If your values are correct and the rest of your image is completely black except for the pink region you can just get all non zero pixels the same way I did for the guy image with np.where(target_img > 0).

Now get just the values of the guy and mask regions

guy = guy_img[bbox_guy[0]:bbox_guy[1], bbox_guy[2]:bbox_guy[3],:]
target = target_img[bbox_mask[0]:bbox_mask[1], bbox_mask[2]:bbox_mask[3],:]

Resize the guy to be the same scale / shape as the mask

guy_h, guy_w, _ = guy.shape
mask_h, mask_w, _ = target.shape
fy = mask_h / guy_h
fx = mask_w / guy_w
scaled_guy = cv2.resize(guy, (0,0), fx=fx,fy=fy)

Reassign the masked region of the target image with the guy image values

for i, row in enumerate(range(bbox_mask[0], bbox_mask[1])):
    for j, col in enumerate(range(bbox_mask[2], bbox_mask[3])):
        target_img[row,col,:] = scaled_guy[i,j,:]

cv2.imshow('', target_img)
cv2.waitKey(0)