Atomic access to shared memory

excalibur picture excalibur · Jan 6, 2012 · Viewed 11.4k times · Source

I have a shared memory between multiple processes that interpets the memory in a certain way. Ex:

DataBlock {
int counter;
double value1;
double ...    }

What I want is for the counter to be updated/incremented atomically. And for a memory release to happen on that address. If I werent using shared memory, for example, it would be something like

std::atomic<int> counter;
atomic_store(counter, newvalue, std::memory_order_release); // perform release     operation on the affected memory location making the write visible to other threads

How do I achieve this for a random memory location (interpreted to be DataBlock counter >above). I can guarantee the address is aligned as required by the architecture (x86 linux)

  1. Make the update atomic - how? (i.e. atomicupdate(addr, newvalue))
  2. Memory syncing for multicore - (i.e. memorysync(addr)) - only way I can see is using the std::atomic_thread_fence(std::memory_order_release) - but this will "establish memory synchronization ordering of ALL atomic and relaxed atomic stores" - thats overkill for me - I just want the counter location to be synchronized. Appreciate any thoughts.

Answer

edA-qa mort-ora-y picture edA-qa mort-ora-y · Jan 6, 2012

I can't answer with authority here, but I can give related information that might help.

  1. Mutexes can be created in shared memory and/or created to be cross-process. Pthread has a special creation flag, I can't remember if that uses shared memory, or you then share a handle. The linux "futex" can use shared memory directly (note the user address may differ, but the underlying real address should be the same)

  2. Hardware atomics work on memory and not process variables. That is, your chip won't care which programs are modifying the variables, the lowest level atomics will thus naturally be cross-process. The same applies to fences.

  3. C++11 fails to specify cross-process atomics. However, if they are lock-free (check the flag) it is hard to see how a compiler could implement them such that cross-process wouldn't work. But you'd be placing a lot of faith in your tool-chain and final platform.

  4. CPU dependency guarantees also track real memory addresses, so as long as your program would be correct in a threaded form it should also be correct in its multi-process form (with respect to visibility).

  5. Kerrek is correct, the abstract machine doesn't really mention multiple processes. However, its synchronization details are written in a way such that they'd equally apply to inter-process as they do to multi-thread. This relates to #3: it'd be hard for a compiler to screw this up.

Short answer, there is no standards compliant way to do this. However, leaning on the way the standard defines mutli-threads there are a lot of assumptions you can make for a quality compiler.

The biggest question is whether an atomic can simply be allocated in shared memory (placement new) and work. Obviously this would only work if it is a true hardware atomic. My guess however is that with a quality compiler/libary the C++ atomics should work find in shared memory.

Have fun verifying behaviour. :)