Pixels in YUV image

Question

I am using opencv to achieve object tracking. I read that YUV image is better option to use than RGB image. My problem is that I fail to understand about the YUV format although i spend much time read notes. Y is the brightness which i believe is calculated from the combination of R, G, B component.

My main problem is how can I access and manipulate the pixels in YUV image format. In RGB format its easy to access the component and therefore change it using simple operatin like

src.at<Vec3b>(j,i).val[0] = 0; for example

But this is not the case in YUV. I need help in accessing and changing the pixel values in YUV image. For example if pixel in RGB is red, then I want to only keep the corresponding pixel in YUV and the rest is removed. Please help me with this.

Answer 1

I would suggest operating on your image in HSV or LAB rather than RGB.

The raw image from the camera will be in YCbCr (sometimes called YUV, which I think is incorrect, but I may be wrong), and laid out in a way that resembles something like YUYV (repeating), so if you can convert directly from that to HSV, you will avoid additional copy and conversion operations which will save you some time. That may only matter to you if you're processing video or batches of images however.

Here's some C++ code for converting between YCbCr and RGB (one uses integer math, the other floating point):

Colour::bgr Colour::YCbCr::toBgrInt() const
{
  int c0 = 22987;
  int c1 = -11698;
  int c2 = -5636;
  int c3 = 29049;

  int y = this->y;
  int cb = this->cb - 128;
  int cr = this->cr - 128;

  int b = y + (((c3 * cb) + (1 << 13)) >> 14);
  int g = y + (((c2 * cb + c1 * cr) + (1 << 13)) >> 14);
  int r = y + (((c0 * cr) + (1 << 13)) >> 14);

  if (r < 0)
    r = 0;
  else if (r > 255)
    r = 255;

  if (g < 0)
    g = 0;
  else if (g > 255)
    g = 255;

  if (b < 0)
    b = 0;
  else if (b > 255)
    b = 255;

  return Colour::bgr(b, g, r);
}

Colour::bgr Colour::YCbCr::toBgrFloat() const
{
  float y = this->y;
  float cb = this->cb;
  float cr = this->cr;

  int r = y + 1.40200 * (cr - 0x80);
  int g = y - 0.34414 * (cb - 0x80) - 0.71414 * (cr - 0x80);
  int b = y + 1.77200 * (cb - 0x80);

  if (r < 0)
    r = 0;
  else if (r > 255)
    r = 255;

  if (g < 0)
    g = 0;
  else if (g > 255)
    g = 255;

  if (b < 0)
    b = 0;
  else if (b > 255)
    b = 255;

  return Colour::bgr(b, g, r);
}

And a conversion from BGR to HSV:

Colour::hsv Colour::bgr2hsv(bgr const& in)
{
  Colour::hsv out;

  int const hstep = 255 / 3;            // Hue step size between red -> green -> blue

  int min = in.r < in.g ? in.r : in.g;
  min = min  < in.b ? min  : in.b;

  int max = in.r > in.g ? in.r : in.g;
  max = max  > in.b ? max  : in.b;

  out.v = max;                          // v
  int chroma = max - min;
  if (max > 0)
  {
    out.s = 255 * chroma / max;         // s
  }
  else
  {
    // r = g = b = 0                    // s = 0, v is undefined
    out.s = 0;
    out.h = 0;
    out.v = 0; // it's now undefined
    return out;
  }

  if (chroma == 0)
  {
    out.h = 0;
    return out;
  }

  const int chroma2 = chroma * 2;
  int offset;
  int diff;

  if (in.r == max)
  {
    offset = 3 * hstep;
    diff = in.g - in.b;
  }
  else if (in.g == max)
  {
    offset = hstep;
    diff = in.b - in.r;
  }
  else
  {
    offset = 2 * hstep;
    diff = in.r - in.g;
  }

  int h = offset + (diff * (hstep + 1)) / chroma2;

  // Rotate such that red has hue 0
  if (h >= 255)
    h -= 255;

  assert(h >= 0 && h < 256);

  out.h = h;

  return out;

Unfortunately I do not have code to do this in one step.

You can also use the built-in OpenCV functions for colour conversion.

cvtColor(img, img, CV_BGR2HSV);