strace to monitor Dockerized application activity

Guillaume Delafosse picture Guillaume Delafosse · Jun 23, 2015 · Viewed 9.9k times · Source

My goal is to monitor which ports are opened and closed by a multi-process application. My plan is to run the application in a Docker container, in order to isolate it, and then use strace to report the application activity.

I've tried with Apache server dockerized :

strace -f -o /tmp/docker.out docker run -D -P apache

I don't see any line in the report file that shows that the application accept a connection in a socket.

Can strace report the activity of processes inside the container?

Answer

Phil E picture Phil E · Sep 6, 2016

The issue with your command+strace combination is that docker has a client/server model, and your docker run represents the client side of a REST API transaction to ask the docker daemon to run the Apache container on your behalf. Depending on how your client is configured, that container may not even run on the same system on which you type your docker run command.

However, to take the simplest case where the Docker client and daemon are on the same system, you can use ps find the PID of the running Apache server and use strace to join and trace the already-started process, as long as that is sufficient for your tracing needs.

Given I had to debug several early-start issues with "runc", the executor for containers in docker version 1.11 and above, I also created a small wrapper for docker-runc which strace's the container process from the start (from the outside system, so strace is not required in the container filesystem). You can find it here on GitHub, although fair warning that it is somewhat buggy for regular use as I believe the shell+strace invocation gets in the way of some signaling between containerd and the real docker-runc and associated processes. A more elegant solution might be to create a variant of runc which knows how to prepend the actual start of the contained process with an strace wrapper rather than intercepting the entire invocation of runc in an strace.