SSE, intrinsics, and alignment

Question

I've written a 3D vector class using a lot of SSE compiler intrinsics. Everything worked fine until I started to instatiate classes having the 3D vector as a member with new. I experienced odd crashes in release mode but not in debug mode and the other way around.

So I read some articles and figured I need to align the classes owning an instance of the 3D vector class to 16 bytes too. So I just added _MM_ALIGN16 (__declspec(align(16)) in front of the classes like so:

_MM_ALIGN16 struct Sphere
{
    // ....

    Vector3 point;
    float radius
};

That seemed to solve the issue at first. But after changing some code my program started to crash in odd ways again. I searched the web some more and found a blog article. I tried what the author, Ernst Hot, did to solve the problem and it works for me too. I added new and delete operators to my classes like this:

_MM_ALIGN16 struct Sphere
{
    // ....

    void *operator new (unsigned int size)
     { return _mm_malloc(size, 16); }

    void operator delete (void *p)
     { _mm_free(p); }

    Vector3 point;
    float radius
};

Ernst mentions that this aproach could be problematic as well, but he just links to a forum which does not exist anymore without explaining why it could be problematic.

So my questions are:

1. What's the problem with defining the operators?

2. Why isn't adding _MM_ALIGN16 to the class definition enough?

3. What's the best way to handle the alignment issues coming with SSE intrinsics?

Answer 1

First of all you have to care for two types of memory allocation:

• Static allocation. For automatic variables to be properly aligned, your type needs a proper alignment specification (e.g. __declspec(align(16)), __attribute__((aligned(16))), or your _MM_ALIGN16). But fortunately you only need this if the alignment requirements given by the type's members (if any) are not sufficient. So you don't need this for you Sphere, given that your Vector3 is already aligned properly. And if your Vector3 contains an __m128 member (which is pretty likely, otherwise I would suggest to do so), then you don't even need it for Vector3. So you usually don't have to mess with compiler specific alignment attributes.

• Dynamic allocation. So much for the easy part. The problem is, that C++ uses, on the lowest level, a rather type-agnostic memory allocation function for allocating any dynamic memory. This only guarantees proper alignment for all standard types, which might happen to be 16 bytes but isn't guaranteed to.

  For this to compensate you have to overload the builtin operator new/delete to implement your own memory allocation and use an aligned allocation function under the hood instead of good old malloc. Overloading operator new/delete is a topic on its own, but isn't that difficult as it might seem at first (though your example is not enough) and you can read about it in this excellent FAQ question.

  Unfortunately you have to do this for each type that has any member needing non-standard alignment, in your case both Sphere and Vector3. But what you can do to make it a bit easier is just make an empty base class with proper overloads for those operators and then just derive all neccessary classes from this base class.

  What most people sometimes tend to forget is that the standard allocator std::alocator uses the global operator new for all memory allocation, so your types won't work with standard containers (and a std::vector<Vector3> isn't that rare a use case). What you need to do is make your own standard conformant allocator and use this. But for convenience and safety it is actually better to just specialize std::allocator for your type (maybe just deriving it form your custom allocator) so that it is always used and you don't need to care for using the proper allocator each time you use a std::vector. Unfortunately in this case you have to again specialize it for each aligned type, but a small evil macro helps with that.

  Additionally you have to look out for other things using the global operator new/delete instead of your custom one, like std::get_temporary_buffer and std::return_temporary_buffer, and care for those if neccessary.

Unfortunately there isn't yet a much better approach to those problems, I think, unless you are on a platform that natively aligns to 16 and know about this. Or you might just overload the global operator new/delete to always align each memory block to 16 bytes and be free of caring for the alignment of each and every single class containing an SSE member, but I don't know about the implications of this approach. In the worst case it should just result in wasting memory, but then again you usually don't allocate small objects dynamically in C++ (though std::list and std::map might think differently about this).

So to sum up:

• Care for proper alignment of static memory using things like __declspec(align(16)), but only if it is not already cared for by any member, which is usually the case.

• Overload operator new/delete for each and every type having a member with non-standard alignment requirements.

• Make a cunstom standard-conformant allocator to use in standard containers of aligned types, or better yet, specialize std::allocator for each and every aligned type.

---

Finally some general advice. Often you only profit form SSE in computation-heavy blocks when performing many vector operations. To simplify all this alignment problems, especially the problems of caring for the alignment of each and every type containing a Vector3, it might be a good aproach to make a special SSE vector type and only use this inside of lengthy computations, using a normal non-SSE vector for storage and member variables.