I've looked online for quite some time and can't find an answer to this. For simplicity, let's just assume I need to fully smooth out the normals of a group of cojoined faces. I want to find the actual geometric bisector between a group of vectors, ignoring duplicate normals and maintaining accuracy with triangle soups. Basically, it needs to:
The most common formula I seem to be finding for normal smoothing is, to simply average them by summing together the normal vectors and divide by three; example:
normalize((A + B + C) / 3);
Of course dividing by three is useless, which already denotes the vibe of naive, off-the-top-of-your-head brute force averaging method that people propose; problems with this are also the fact that it messes up big time with even triangle soups and parallel normals.
Another remark I seem to find is to keep the initial "facet" normals as they come from a common cross multiply operation that generates them, as that way they are (kinda) weighted against the triangle's area. This might be something you want to do in some cases, however I need the pure bisector, so area must not influence the formula, and even accounting for it it still gets messed up by triangle soups.
I saw one mentioned method that says to weight against the angle between the adjacent faces, but I can't seem to implement the formula correctly - either that or it's not doing what I want. However that's hard for me to say, since I can't find a concise explanation for it and my mind is becoming numb from all this wasted brainstorming.
Anybody knows of a general formula? If it is of any help, I'm working with C++ and DirectX11.
Edit: Here's some similar questions that describe some of the methods;
Also this article: http://www.bytehazard.com/articles/vertnorm.html
Unfortunately, the implementations I tried didn't work and I couldn't find a clear, concise statement about which formula was actually the one that I needed. After some trial and error I finally figured out that weighting by angle was the correct way to do it, only I wasn't able to implement it correctly; as it seems to be working now, I'll add my implementation as an answer below.
The correct method is to weight the "facet" normals against the angle between the two neighbor vertices of the one shared in the edge/corner.
(here shown the angle with respect to each face)
Here's a rundown of an example implementation:
for (int f = 0; f < tricount; f++)
{
// ...
// p1, p2 and p3 are the points in the face (f)
// calculate facet normal of the triangle using cross product;
// both components are "normalized" against a common point chosen as the base
float3 n = (p2 - p1).Cross(p3 - p1); // p1 is the 'base' here
// get the angle between the two other points for each point;
// the starting point will be the 'base' and the two adjacent points will be normalized against it
a1 = (p2 - p1).Angle(p3 - p1); // p1 is the 'base' here
a2 = (p3 - p2).Angle(p1 - p2); // p2 is the 'base' here
a3 = (p1 - p3).Angle(p2 - p3); // p3 is the 'base' here
// normalize the initial facet normals if you want to ignore surface area
if (!area_weighting)
{
normalize(n);
}
// store the weighted normal in an structured array
v1.wnormals.push_back(n * a1);
v2.wnormals.push_back(n * a2);
v3.wnormals.push_back(n * a3);
}
for (int v = 0; v < vertcount; v++)
{
float3 N;
// run through the normals in each vertex's array and interpolate them
// vertex(v) here fetches the data of the vertex at index 'v'
for (int n = 0; n < vertex(v).wnormals.size(); v++)
{
N += vertex(v).wnormals.at(n);
}
// normalize the final normal
normalize(N);
}
Here's an example of a "naive" average of the normals (i.e. without angle weighting);
You can see the facet components being all the same, but since some sides have two faces, their part of the interpolation is doubled, making the average skewed. Weighting only against the surface area, but not the angles, produces similar results.
This is instead the same model, but with angle weighting enabled;
Now the interpolated normals are all geometrically correct.