How do I use `setrlimit` to limit memory usage? RLIMIT_AS kills too soon; RLIMIT_DATA, RLIMIT_RSS, RLIMIT_STACK kill not at all

Question

I'm trying to use setrlimit to limit my memory usage on a Linux system, in order to stop my process from crashing the machine (my code was crashing nodes on a high performance cluster, because a bug led to memory consumption in excess of 100 GiB). I can't seem to find the correct resource to pass to setrlimit; I think it should be resident, which cannot be limited with setrlimit, but I am confused by resident, heap, stack. In the code below; if I uncomment only RLIMIT_AS, the code fails with MemoryError at numpy.ones(shape=(1000, 1000, 10), dtype="f8") even though that array should be only 80 MB. If I uncomment only RLIMIT_DATA, RLIMIT_RSS, or RLIMIT_STACK both arrays get allocated successfully, even though the total memory usage is 2 GB, or twice the desired maximum.

I would like to make make my program fail (no matter how) as soon as it tries to allocate too much RAM. Why do none of RLIMIT_DATA, RLIMIT_RSS, RLIMIT_STACK and RLIMIT_AS do what I mean, and what is the correct resource to pass to setrlimit?

$ cat mwe.py 
#!/usr/bin/env python3.5

import resource
import numpy

#rsrc = resource.RLIMIT_AS
#rsrc = resource.RLIMIT_DATA
#rsrc = resource.RLIMIT_RSS
#rsrc = resource.RLIMIT_STACK

soft, hard = resource.getrlimit(rsrc)
print("Limit starts as:", soft, hard)

resource.setrlimit(rsrc, (1e9, 1e9))

soft, hard = resource.getrlimit(rsrc)
print("Limit is now:", soft, hard)
print("Allocating 80 KB, should certainly work")
M1 = numpy.arange(100*100, dtype="u8")

print("Allocating 80 MB, should work")
M2 = numpy.arange(1000*1000*10, dtype="u8")

print("Allocating 2 GB, should fail")
M3 = numpy.arange(1000*1000*250, dtype="u8")

input("Still here…")

Output with the RLIMIT_AS line uncommented:

$ ./mwe.py 
Limit starts as: -1 -1
Limit is now: 1000000000 -1
Allocating 80 KB, should certainly work
Allocating 80 MB, should work
Traceback (most recent call last):
  File "./mwe.py", line 22, in <module>
    M2 = numpy.arange(1000*1000*10, dtype="u8")
MemoryError

Output when running with any of the other ones uncommented:

$ ./mwe.py 
Limit starts as: -1 -1
Limit is now: 1000000000 -1
Allocating 80 KB, should certainly work
Allocating 80 MB, should work
Allocating 2 GB, should fail
Still here…

At the final line, top reports that my process is using 379 GB VIRT, 2.0 GB RES.

---

System details:

$ uname -a
Linux host.somewhere.ac.uk 2.6.32-573.3.1.el6.x86_64 #1 SMP Mon Aug 10 09:44:54 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux

$ cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 6.7 (Santiago)

$ free -h
             total       used       free     shared    buffers     cached
Mem:          2.0T       1.9T        37G       1.6G       3.4G       1.8T
-/+ buffers/cache:        88G       1.9T 
Swap:         464G       4.8M       464G 

$ python3.5 --version
Python 3.5.0

$ python3.5 -c "import numpy; print(numpy.__version__)"
1.11.1

Answer 1

Alas I have no answer for your question. But I hope the following might help:

• Your script works as expected on my system. Please share exact spec for yours, might be there is a known problem with Linux distro, kernel or even numpy...
• You should be OK with RLIMIT_AS. As explained here this should limit the entire virtual memory used by the process. And virtual memory includes all: swap memory, shared libraries, code and data. More details here.

• You may add the following function (adopted from this answer) to your script to check actual virtual memory usage at any point:

  def peak_virtual_memory_mb():
      with open('/proc/self/status') as f:
          status = f.readlines()
          vmpeak = next(s for s in status if s.startswith("VmPeak:"))
          return vmpeak
  

• A general advice, disable swap memory. In my experience with high performance servers it does more harm than solves problems.