How to create Stencil buffer with texture (Image) in OpenGL-ES 2.0

infiniteLoop picture infiniteLoop · Sep 27, 2011 · Viewed 8.6k times · Source

Can I have Stencil prepared with a texture (Image) in OpenGL 2.0 So that some part of the Image will be transparent and as a result it will be transfered as is
to Stencil buffer and then will use this use this stencil buffer for further drawing.


EDIT by datenwolf to account for OPs question update in a answer:

By @InfiniteLoop:

@datenwolf thanks a lot for ur reply but no success :( here is my code

- (void)render 
{
    [EAGLContext setCurrentContext:context];
    glBindFramebuffer(GL_FRAMEBUFFER, defaultFramebuffer);
    glUseProgram(program);
    glClearColor(0, 104.0/255.0, 55.0/255.0, 1.0);
    glViewport(0, 0, backingWidth, backingHeight);
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);
    // Set the stencil clear value
    glClearStencil ( 0x1 );

    // Set the depth clear value
    glClearDepthf( 0.75f );



    ////////
    ////
    GLfloat threshold = 0.5f;//half transparency
    /* the order in which orthogonal OpenGL state is set doesn't matter */
    glEnable(GL_STENCIL_TEST);
    glEnable(GL_ALPHA_TEST);

    /* don't write to color or depth buffer */
    glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
    glDepthMask(GL_FALSE);

    glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);

    /* first set alpha test so that fragments greater than the threshold
     will pass thus will set nonzero bits masked by 1 in stencil */
    glStencilFunc(GL_ALWAYS, 1, 1);//set one
    glAlphaFunc(GL_GREATER, threshold);//greater than half transparency

    glVertexAttribPointer(positionLoc, 3, GL_FLOAT, GL_FALSE, 0, fullVertices);
    glVertexAttribPointer(colorLoc, 4, GL_FLOAT, GL_FALSE, 0, colorVertices);
    glVertexAttribPointer(textureLoc, 2, GL_FLOAT, GL_FALSE, 0, textureVertices);
    [self drawTexture:texture2DMaskImage];
    /* second pass of the fragments less or equal than the threshold
     will pass thus will set zero bits masked by 1 in stencil */
    glStencilFunc(GL_ALWAYS, 0, 1);//set zero
    glAlphaFunc(GL_LEQUAL, threshold);//Less than Half transparency
    glVertexAttribPointer(positionLoc, 3, GL_FLOAT, GL_FALSE, 0, fullVertices);
    glVertexAttribPointer(colorLoc, 4, GL_FLOAT, GL_FALSE, 0, colorVertices);
    glVertexAttribPointer(textureLoc, 2, GL_FLOAT, GL_FALSE, 0, textureVertices);
    [self drawTexture:texture2DMaskImage];


// till here as suggested by u after this I have added to draw
// the next Image which will use the created Stencil
    glDepthMask(GL_TRUE);
    glDisable(GL_ALPHA_TEST);
    glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
    //
    glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);//stop further write to stencil buffer
    glStencilFunc(GL_EQUAL, 0, 1);//pass if 0
    glVertexAttribPointer(positionLoc, 3, GL_FLOAT, GL_FALSE, 0, fullVertices);
    glVertexAttribPointer(colorLoc, 4, GL_FLOAT, GL_FALSE, 0, colorVertices);
    glVertexAttribPointer(textureLoc, 2, GL_FLOAT, GL_FALSE, 0, textureVertices);
    [self drawTexture:texture2D];
    ////
    ///////



    // Validate program before drawing. This is a good check, 
    // but only really necessary in a debug build.
    // DEBUG macro must be defined in your debug configurations
    //  if that's not already the case.
#if defined(DEBUG)
    if (![self validateProgram:program])
    {
        NSLog(@"Failed to validate program: %d", program);
        return;
    }
#endif
    // draw
    [context presentRenderbuffer:GL_RENDERBUFFER];//Present
}

- (void) drawTexture:(Texture2D*)texture
{
    // handle viewing matrices
    GLfloat proj[16], modelview[16], modelviewProj[16];
    // setup projection matrix (orthographic)
    mat4f_LoadOrtho(-1.0f, 1.0f, -1.0f, 1.0f, -1.0f, 1.0f, proj);
    //   glUniformMatrix4fv(uniforms[UNIFORM_PROJECTION], 1, 0, proj);
    // setup modelview matrix (rotate around z)
    mat4f_LoadIdentity(modelview);
    mat4f_LoadZRotation(0.0, modelview);
    mat4f_LoadTranslation3D(0.0, 0.0, 0.0, modelview);
    // projection matrix * modelview matrix
    mat4f_MultiplyMat4f(proj, modelview, modelviewProj);
    glUniformMatrix4fv(_modelViewUniform, 1, 0, modelviewProj);
    // update uniform values
    glActiveTexture(GL_TEXTURE0);
    glBindTexture(GL_TEXTURE_2D, texture.name);
    glUniform1i(_textureUniform, 0);
    glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_SHORT, cubeIndices);
}

please shed some light I am in desperate need :( thanks once again.

Answer

datenwolf picture datenwolf · Sep 27, 2011

Yes, you can use alpha testing (glEnable(GL_ALPHA_TEST); glAlphaFunc(COMPARISION, VALUE);) to discard fragments; discarded fragments will not be stencil tested, so you can use the passing fragments to build the stencil mask.

EDIT code example

/* the order in which orthogonal OpenGL state is set doesn't matter */
glEnable(GL_STENCIL_TEST);
glEnable(GL_ALPHA_TEST);

/* don't write to color or depth buffer */
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
glDepshMask(GL_FALSE);

glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);

/* first set alpha test so that fragments greater than the threshold
   will pass thus will set nonzero bits masked by 1 in stencil */
glStencilFunc(GL_ALWAYS, 1, 1);
glAlphaFunc(GL_GREATER, threshold);
draw_textured_primitive();


/* second pass of the fragments less or equal than the threshold
   will pass thus will set zero bits masked by 1 in stencil */
glStencilFunc(GL_ALWAYS, 0, 1);
glAlphaFunc(GL_LEQUAL, threshold);
draw_textured_primitive();

Don't forget to revert OpenGL state for drawing afterwards.

EDIT to account for updated question:

Since you're actually using iOS, thus OpenGL-ES 2.0 things get a bit different. There's no alpha test in OpenGL-ES2. Instead you're expected to discard the fragment in the fragment shader, if it's below/above a chosen threshold, by using the discard keyword/statement.