Docker build from Dockerfile with more memory

docker memory docker-build neural-mt
alvas picture alvas · Jul 28, 2017 · Viewed 19.1k times · Source

How to docker build from Dockerfile with more memory?

This is a different question from this Allow more memory when docker build a Dockerfile

When installing the software natively, there is enough memory to successfully build and install the marian tool

But when building the Docker image using the Dockerfile https://github.com/marian-nmt/marian/blob/master/scripts/docker/Dockerfile.cpu , it fails with multiple memory exhausted errors 

virtual memory exhausted: Cannot allocate memory

[out]:

Step : RUN cmake $MARIANPATH && make -j
 ---> Running in 4867d166d17a
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CUDA_TOOLKIT_ROOT_DIR not found or specified
-- Cannot find CUDA libraries. Compiling without them.
-- Could NOT find CUDA (missing:  CUDA_TOOLKIT_ROOT_DIR CUDA_NVCC_EXECUTABLE CUDA_INCLUDE_DIRS CUDA_CUDART_LIBRARY) 
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Boost version: 1.58.0
-- Found the following Boost libraries:
--   system
--   filesystem
--   program_options
--   timer
--   iostreams
--   python
--   thread
--   chrono
--   regex
--   date_time
--   atomic
-- Found Python
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython2.7.so (found suitable version "2.7.12", minimum required is "2.7") 
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.8") 
-- Found Git: /usr/bin/git (found version "2.7.4") 
-- Git version: 5abc774
-- Found SparseHash: /usr/include  
-- Configuring done
-- Generating done
-- Build files have been written to: /marian/build
Scanning dependencies of target fast_align
Scanning dependencies of target extract_lex
Scanning dependencies of target libcnpy
Scanning dependencies of target atools
Scanning dependencies of target libyaml-cpp-amun
[  2%] Building CXX object src/amun/3rd_party/extract_lex/CMakeFiles/extract_lex.dir/exception.cpp.o
[  2%] Building CXX object src/amun/3rd_party/extract_lex/CMakeFiles/extract_lex.dir/utils.cpp.o
[  3%] Building CXX object src/amun/3rd_party/fast_align/CMakeFiles/fast_align.dir/src/ttables.cc.o
[  6%] Building CXX object src/amun/3rd_party/fast_align/CMakeFiles/fast_align.dir/src/fast_align.cc.o
[  7%] Building CXX object src/amun/3rd_party/extract_lex/CMakeFiles/extract_lex.dir/extract-lex-main.cpp.o
[ 10%] Building CXX object src/amun/3rd_party/CMakeFiles/libcnpy.dir/cnpy/cnpy.cpp.o
[ 10%] Building CXX object src/amun/3rd_party/fast_align/CMakeFiles/atools.dir/src/atools.cc.o
[ 10%] Building CXX object src/amun/3rd_party/fast_align/CMakeFiles/atools.dir/src/alignment_io.cc.o
[ 11%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/regex_yaml.cpp.o
[ 12%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/scanner.cpp.o
[ 14%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/convert.cpp.o
[ 15%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/stream.cpp.o
[ 16%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/memory.cpp.o
[ 17%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/node.cpp.o
[ 19%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/directives.cpp.o
[ 20%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/null.cpp.o
[ 21%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/emitfromevents.cpp.o
[ 23%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/singledocparser.cpp.o
[ 24%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/emitterstate.cpp.o
[ 25%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/scantag.cpp.o
[ 26%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/simplekey.cpp.o
[ 28%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/binary.cpp.o
[ 29%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/parser.cpp.o
[ 30%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/nodeevents.cpp.o
[ 32%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/emit.cpp.o
[ 33%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/exp.cpp.o
[ 35%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/nodebuilder.cpp.o
[ 38%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/scanscalar.cpp.o
[ 39%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/emitter.cpp.o
[ 41%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/tag.cpp.o
[ 42%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/scantoken.cpp.o
[ 43%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/ostream_wrapper.cpp.o
Scanning dependencies of target libcommon
[ 37%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/node_data.cpp.o
[ 44%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/contrib/graphbuilder.cpp.o
[ 46%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/emitterutils.cpp.o
[ 34%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/contrib/graphbuilderadapter.cpp.o
[ 47%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/parse.cpp.o
[ 48%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/scorer.cpp.o
[ 50%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/git_version.cpp.o
[ 51%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/base_matrix.cpp.o
[ 52%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/sentence.cpp.o
[ 53%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/config.cpp.o
[ 55%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/utils.cpp.o
[ 56%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/vocab.cpp.o
[ 65%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/filter.cpp.o
[ 66%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/god.cpp.o
[ 67%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/exception.cpp.o
[ 69%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/search.cpp.o
[ 69%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/hypothesis.cpp.o
[ 70%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/translation_task.cpp.o
[ 70%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/printer.cpp.o
[ 70%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/sentences.cpp.o
[ 70%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/history.cpp.o
[ 70%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/logging.cpp.o
[ 70%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/processor/bpe.cpp.o
[ 71%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/output_collector.cpp.o
[ 73%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/types.cpp.o
[ 74%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/loader.cpp.o
virtual memory exhausted: Cannot allocate memory
virtual memory exhausted: Cannot allocate memory
virtual memory exhausted: Cannot allocate memory

And eventually it leads to:

src/amun/CMakeFiles/libcommon.dir/build.make:254: recipe for target 'src/amun/CMakeFiles/libcommon.dir/common/loader.cpp.o' failed
virtual memory exhausted: Cannot allocate memory
make[2]: *** [src/amun/CMakeFiles/cpumode.dir/cpu/decoder/encoder_decoder.cpp.o] Error 1
src/amun/CMakeFiles/cpumode.dir/build.make:110: recipe for target 'src/amun/CMakeFiles/cpumode.dir/cpu/decoder/encoder_decoder.cpp.o' failed
[ 79%] Built target libcnpy
virtual memory exhausted: Cannot allocate memory
src/amun/CMakeFiles/libcommon.dir/build.make:326: recipe for target 'src/amun/CMakeFiles/libcommon.dir/common/printer.cpp.o' failed
make[2]: *** [src/amun/CMakeFiles/libcommon.dir/common/printer.cpp.o] Error 1
CMakeFiles/Makefile2:340: recipe for target 'src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/all' failed
make[1]: *** [src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/all] Error 2
CMakeFiles/Makefile2:182: recipe for target 'src/amun/CMakeFiles/libcommon.dir/all' failed
make[1]: *** [src/amun/CMakeFiles/libcommon.dir/all] Error 2
make[1]: *** [src/amun/CMakeFiles/cpumode.dir/all] Error 2
CMakeFiles/Makefile2:110: recipe for target 'src/amun/CMakeFiles/cpumode.dir/all' failed
make: *** [all] Error 2

Looking at the docker build --help, there are options to control memory usage:

$ docker build --help

Usage:  docker build [OPTIONS] PATH | URL | -

Build an image from a Dockerfile

Options:
      --build-arg list             Set build-time variables (default [])
      --cache-from stringSlice     Images to consider as cache sources
      --cgroup-parent string       Optional parent cgroup for the container
      --compress                   Compress the build context using gzip
      --cpu-period int             Limit the CPU CFS (Completely Fair Scheduler) period
      --cpu-quota int              Limit the CPU CFS (Completely Fair Scheduler) quota
  -c, --cpu-shares int             CPU shares (relative weight)
      --cpuset-cpus string         CPUs in which to allow execution (0-3, 0,1)
      --cpuset-mems string         MEMs in which to allow execution (0-3, 0,1)
      --disable-content-trust      Skip image verification (default true)
  -f, --file string                Name of the Dockerfile (Default is 'PATH/Dockerfile')
      --force-rm                   Always remove intermediate containers
      --help                       Print usage
      --isolation string           Container isolation technology
      --label list                 Set metadata for an image (default [])
  -m, --memory string              Memory limit
      --memory-swap string         Swap limit equal to memory plus swap: '-1' to enable unlimited swap
      --network string             Set the networking mode for the RUN instructions during build (default "default")
      --no-cache                   Do not use cache when building the image
      --pull                       Always attempt to pull a newer version of the image
  -q, --quiet                      Suppress the build output and print image ID on success
      --rm                         Remove intermediate containers after a successful build (default true)
      --security-opt stringSlice   Security options
      --shm-size string            Size of /dev/shm, default value is 64MB
  -t, --tag list                   Name and optionally a tag in the 'name:tag' format (default [])
      --ulimit ulimit              Ulimit options (default [])

But I couldn't figure out the correct syntax of where exactly to put the -m option -_-|||

It isn't before the Dockerfile:

# Before Docker file.
$ docker build -m 4g Dockerfile.cpu -t ibot-cpu .

"docker build" requires exactly 1 argument(s).
See 'docker build --help'.

Usage:  docker build [OPTIONS] PATH | URL | -

Build an image from a Dockerfile

It isn't after Dockerfile before -t

# Before -t
$ docker build Dockerfile.cpu -m 4g -t ibot-cpu .

"docker build" requires exactly 1 argument(s).
See 'docker build --help'.

Usage:  docker build [OPTIONS] PATH | URL | -

Build an image from a Dockerfile

It isn't after -t before the local path 

# Before local path
$ docker build Dockerfile.cpu -t ibot-cpu -m 4g .

"docker build" requires exactly 1 argument(s).
See 'docker build --help'.

Usage:  docker build [OPTIONS] PATH | URL | -

Build an image from a Dockerfile

It isn't after the local path at the end too...

# At the end...
$ docker build Dockerfile.cpu -t ibot-cpu . -m 4g

"docker build" requires exactly 1 argument(s).
See 'docker build --help'.

Usage:  docker build [OPTIONS] PATH | URL | -

Build an image from a Dockerfile

How to docker build from Dockerfile with more memory?

My docker version:

docker version
Client:
 Version:      17.03.1-ce
 API version:  1.27
 Go version:   go1.7.5
 Git commit:   c6d412e
 Built:        Tue Mar 28 00:40:02 2017
 OS/Arch:      darwin/amd64

Server:
 Version:      17.04.0-ce
 API version:  1.28 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   4845c56
 Built:        Wed Apr  5 18:45:47 2017
 OS/Arch:      linux/amd64
 Experimental: false

Answer

Robert picture Robert · Jul 31, 2017

It is not something about order. The Dockerfile must be specified with -f

docker build -f Dockerfile.cpu -t ibot-cpu -m 4g .

However, take into account that by default docker does not limit the container memory. It can take the whole free memory.

As I can see that you are on OSX, which runs docker over a Linux VM. Configure the max memory clicking the whale icon in the task bar. It's 2GB by default.

For further information please see my other answer: How to assign more memory to docker container

Related questions

Docker error : no space left on device

I installed docker on a Debian 7 machine in the following way $ echo deb http://get.docker.io/ubuntu docker main > /etc/apt/sources.list.d/docker.list $ sudo apt-get update $ curl -sSL https://get.docker.com/ubuntu/ | sudo sh …

Read More
Docker container and memory consumption

Assume I am starting a big number of docker containers which are based on the same docker image. It means that each docker container is running the same application. It could be the case that the application is big enough …

Read More
Java using much more memory than heap size (or size correctly Docker memory limit)

For my application, the memory used by the Java process is much more than the heap size. The system where the containers are running starts to have memory problem because the container is taking much more memory than the heap …

Read More