glLineStipple deprecated in OpenGL 3.1

Question

glLineStipple has been deprecated in the latest OpenGL APIs. What is it replaced with? If not replaced, how can I get a similar effect? (I don't want to use a compatibility profile of course...)

Answer 1

Sorry, it hasn't been replaced with anything. The first idea coming to my mind for emulating it would be the geometry shader. You feed the geometry shader with a line, compute its screen space length and based on that you generate a variable number of sub lines between its start and end vertex.

---

EDIT: Perhaps you could also use a 1D texture with the alpha (or red) channel encoding the pattern as 0.0 (no line) or 1.0 (line) and then have the lines texture coordinate go from 0 to 1 and in the fragment chader you make a simple alpha test, discarding fragments with alpha below some threshold. You can facilitate the geometry shader to generate your line texCoords, as otherwise you need different vertices for every line. This way you can also make the texCoord dependent on the screen space length of the line.

The whole thing get's more difficult if you draw triangles (using polygon mode GL_LINE). Then you have to do the triangle-line transformation yourself in the geometry shader, putting in triangles and putting out lines (that could also be a reason for deprecating polygon mode in the future, if it hasn't already).

---

EDIT: Although I believe this question abandomned, I have made a simple shader triple for the second approach. It's just a minimal solution, feel free to add custom features yourself. I haven't tested it because I lack the neccessary hardware, but you should get the point:

uniform mat4 modelViewProj;

layout(location=0) in vec4 vertex;

void main()
{
    gl_Position = modelViewProj * vertex;
}

The vertex shader is a simple pass through.

layout(lines) in;
layout(line_strip, max_vertices=2) out;

uniform vec2 screenSize;
uniform float patternSize;

noperspective out float texCoord;

void main()
{
    vec2 winPos0 = screenSize.xy * gl_in[0].gl_Position.xy / gl_in[0].gl_Position.w;
    vec2 winPos1 = screenSize.xy * gl_in[1].gl_Position.xy / gl_in[1].gl_Position.w;
    gl_Position = gl_in[0].gl_Position;
    texCoord = 0.0;
    EmitVertex();
    gl_Position = gl_in[1].gl_Position;
    texCoord = 0.5 * length(winPos1-winPos0) / patternSize;
    EmitVertex();
}

In the geometry shader we take a line and compute its screen space length in pixels. We then devide this by the size of the stipple pattern texture, which would be factor*16 when emulating a call to glLineStipple(factor, pattern). This is taken as 1D texture coordinate of the second line end point.

Note that this texture coordinate has to be interpolated linearly (noperspective interpolation specifier). The usual perpective-correct interpolation would cause the stipple pattern to "squeeze together" on farther away parts of the line, whereas we are explicitly working with screen-space values.

uniform sampler1D pattern;
uniform vec4 lineColor;

noperspective in float texCoord;

layout(location=0) out vec4 color;

void main()
{
    if(texture(pattern, texCoord).r < 0.5)
        discard;
    color = lineColor;
}

The fragment shader now just performs a simple alpha test using the value from the pattern texture, which contains a 1 for line and a 0 for no line. So to emulate the fixed function stipple you would have a 16 pixel 1-component 1D texture instead of a 16bit pattern. Don't forget to set the pattern's wrapping mode to GL_REPEAT, about the filtering mode I'm not that sure, but I suppose GL_NEAREST would be a good idea.

But as said earlier, if you want to render triangles using glPolygonMode, it won't work this way. Instead you have to adapt the geometry shader to accept triangles and generate 3 lines for each triangle.

---

EDIT: In fact OpenGL 3's direct support for integer operations in shaders allows us to completely drop this whole 1D-texture approach and work straight-forward with an actual bit-pattern. Thus the geometry shader is slightly changed to put out the actual screen-size pattern coordinate, without normalization:

texCoord = 0.5 * length(winPos1-winPos0);

In the fragment shader we then just take a bit pattern as unsigned integer (though 32-bit in contrast to glLineStipple's 16-bit value) and the stretch factor of the pattern and just take the texture coordinate (well, no texture anymore actually, but nevermind) modulo 32 to get it's position on the pattern (those explicit uints are annoying, but my GLSL compiler says implicit conversions between int and uint are evil):

uniform uint pattern;
uniform float factor;

...
uint bit = uint(round(linePos/factor)) & 31U;
if((pattern & (1U<<bit)) == 0U)
    discard;