Docker-compose: node_modules not present in a volume after npm install succeeds

Question

I have an app with the following services:

• web/ - holds and runs a python 3 flask web server on port 5000. Uses sqlite3.
• worker/ - has an index.js file which is a worker for a queue. the web server interacts with this queue using a json API over port 9730. The worker uses redis for storage. The worker also stores data locally in the folder worker/images/

Now this question only concerns the worker.

worker/Dockerfile

FROM node:0.12

WORKDIR /worker

COPY package.json /worker/
RUN npm install

COPY . /worker/

docker-compose.yml

redis:
    image: redis
worker:
    build: ./worker
    command: npm start
    ports:
        - "9730:9730"
    volumes:
        - worker/:/worker/
    links:
        - redis

When I run docker-compose build, everything works as expected and all npm modules are installed in /worker/node_modules as I'd expect.

npm WARN package.json [email protected] No README data

> [email protected] install /worker/node_modules/pageres/node_modules/screenshot-stream/node_modules/phantom-bridge/node_modules/phantomjs
> node install.js

<snip>

But when I do docker-compose up, I see this error:

worker_1 | Error: Cannot find module 'async'
worker_1 |     at Function.Module._resolveFilename (module.js:336:15)
worker_1 |     at Function.Module._load (module.js:278:25)
worker_1 |     at Module.require (module.js:365:17)
worker_1 |     at require (module.js:384:17)
worker_1 |     at Object.<anonymous> (/worker/index.js:1:75)
worker_1 |     at Module._compile (module.js:460:26)
worker_1 |     at Object.Module._extensions..js (module.js:478:10)
worker_1 |     at Module.load (module.js:355:32)
worker_1 |     at Function.Module._load (module.js:310:12)
worker_1 |     at Function.Module.runMain (module.js:501:10)

Turns out none of the modules are present in /worker/node_modules (on host or in the container).

If on the host, I npm install, then everything works just fine. But I don't want to do that. I want the container to handle dependencies.

What's going wrong here?

(Needless to say, all packages are in package.json.)

Answer 1

This happens because you have added your worker directory as a volume to your docker-compose.yml, as the volume is not mounted during the build.

When docker builds the image, the node_modules directory is created within the worker directory, and all the dependencies are installed there. Then on runtime the worker directory from outside docker is mounted into the docker instance (which does not have the installed node_modules), hiding the node_modules you just installed. You can verify this by removing the mounted volume from your docker-compose.yml.

A workaround is to use a data volume to store all the node_modules, as data volumes copy in the data from the built docker image before the worker directory is mounted. This can be done in the docker-compose.yml like this:

redis:
    image: redis
worker:
    build: ./worker
    command: npm start
    ports:
        - "9730:9730"
    volumes:
        - ./worker/:/worker/
        - /worker/node_modules
    links:
        - redis

I'm not entirely certain whether this imposes any issues for the portability of the image, but as it seems you are primarily using docker to provide a runtime environment, this should not be an issue.

If you want to read more about volumes, there is a nice user guide available here: https://docs.docker.com/userguide/dockervolumes/

EDIT: Docker has since changed it's syntax to require a leading ./ for mounting in files relative to the docker-compose.yml file.