Using mmap and madvise for huge pages

Question

I want to allocate memory on the hugepages being used by a Linux machine. I see that there are two ways to do this, using mmap and madvise.

That is, using the MAP_HUGETLB flag with the mmap call -

base_ptr_ = mmap(NULL, memory_size_, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB, -1, 0);

And the MADV_HUGEPAGE flag with the madvise call -

madvise(base_ptr_, memory_size_, MADV_HUGEPAGE);

Could someone explain the difference between the two?

Answer 1

Both functions perform different operations, which may or may not matter in your context:

• madvise sets a flag for all the memory mappings corresponding to the region passed to it, telling the khugepaged kernel thread that it can consider said mappings for promotion to huge pages. That will only work if transparent hugepage support is enabled (The status of transparent hugepage support is available under /sys/kernel/mm/transparent_hugepage/enabled), which will be the case in most distros, but may be disabled on embedded systems.

• mmap will actually go ahead and reserve the pages from the kernel's internal hugetlbfs mount, whose status can be seen under /sys/kernel/mm/hugepages. The pages in question need to be available by the time mmap is invoked (see HugePages_Free in /proc/meminfo), or mmap will fail.

The two mechanisms have their own doc file in the kernel tree: hugetlbpage.txt and transhuge.txt