176

What is the best way to find a specific string in the logs of a docker container? Let's say I want to see all requests, that are made in the "nginx" docker image that came from a ip starting with "127."

grep wont work as expected on docker logs command:

docker logs nginx | grep "127."

It prints all logs, but does not filter the result!

Brad Koch
  • 19,267
  • 19
  • 110
  • 137
Gering
  • 3,010
  • 3
  • 21
  • 23
  • The Question is: What is the best way to find a specific string in the logs of an docker container – Gering Jan 11 '16 at 15:54
  • Does `docker logs` send output to standard output? Because if it does then `grep` should work just fine. If not then it is a bit busted and you'll need to redirect standard error to standard output before filtering with `grep`. – Etan Reisner Jan 11 '16 at 16:17
  • 3
    check stderr et atdout, extract from https://github.com/docker/docker/issues/7440 `$ docker run -d --name foo busybox ls abcd $ docker logs foo > stdout.log 2>stderr.log $ cat stdout.log $ cat stderr.log ls: abcd: No such file or directory` – user2915097 Jan 11 '16 at 16:46
  • 1
    @Robse Sorry, your question was hard to get before you added that example. Looks like `docker logs` is hard-to-grep since it contains terminal control chars. I would grep trough the nginx log files. – hek2mgl Jan 11 '16 at 19:09
  • wonder if you could do this with --follow so the grep keeps looping as containers are being initialized – Mr-Programs Sep 01 '19 at 20:12
  • More info on `docker logs` output to stdout and stderr: https://github.com/moby/moby/issues/41139 – cdalxndr Jun 22 '20 at 11:05

11 Answers11

333

this can happen if the container is logging to stderr, piping works only for stdout, so try:

docker logs nginx 2>&1 | grep "127."

"2>&1" will tells the shell to redirect stderr to stdout and give the output to grep using pipe.

Edoardo
  • 4,485
  • 1
  • 27
  • 31
36

As vim fan I prefer to use less and search with / (or ? for backwards search)

docker logs nginx 2>&1 | less

More about vim search here

Nedvajz
  • 891
  • 8
  • 13
14
docker logs <container_name> 2>&1 | grep <string>
miigotu
  • 1,387
  • 15
  • 15
11

Additionally, I found it usefull to highlight some terms that I'm searching for in the logs. Especially on productive installations where a lot of log output is generated. In my case I want to highlight COUNT(*) statements. But with a simple grep I can't see the entire SQL statement since it's a multi line statement. This is possible with -E switch and some regex:

For example, the following snippet search for all queries that contain COUNT(*) as well as count(*):

docker logs <containerName> -f | grep --line-buffered -i -E --color "select count\(\*\)|$"

Some explanation:

  • docker logs -f tell docker to follow the logs, so the filter applys also to new entrys like when viewing it using tail -f
  • greps --line-buffered switch flushes the output on every line, which is required to grep in real time when docker logs -f is used
  • -E is an extended regex pattern, required to apply our pattern that allow us returning also the non matching results
  • --color highlights the matched parts (seems the default behaviour on my Ubuntu 16.04 LTS, but maybe not on other distributions, so I included it here to be safe)
  • * is escaped to disable its special glob functionality, where (, and ) are masked to avoid their regex meaning as group, which is enabled by -E switch

If the container logs to stderr, you can pipe them as Edoardo already wrote for a simple grep:

docker logs <containerName> -f 2>&1 | grep --line-buffered -i -E --color "select count\(\*\)|$"

The -f switch could be omitted if no live grep is wanted. In both cases, you see the entire log buth with highlighted search term like this:

enter image description here

Lion
  • 16,606
  • 23
  • 86
  • 148
11

You could also use an anonymous pipe:

docker logs nginx 2> >(grep '127.')
karlsebal
  • 1,449
  • 17
  • 23
  • 1
    For some reason, this is the only answer that works for me. The others don't work, I can't lay my finger on it. It seems as if ```docker logs``` writes directly to the screen, bypassing STDOUT or STDERR. – Anne van Leyden Apr 08 '21 at 06:17
7

To follow up on the comments and clarify this for anyone else hitting this issue. Here is the simplest way I can see to search an nginx container log.

docker logs nginx > stdout.log 2>stderr.log
cat stdout.log | grep "127."

IMO its kinda messy because you need to create and delete these potentially very large files. Hopefully we'll get some tooling to make it a bit more convenient.

jackkav
  • 574
  • 4
  • 13
7

you can use these:

  1. On all shells: 
    docker logs nginx 2>&1 | grep "127."
  2. Only on newer versions of bash works: 
    docker logs nginx | &> grep "127"

What is this syntax?

  • Standard input (0)
  • Standard output (1)
  • Standard error (2)

The > operator actually defaults to using 1 as the file descriptor number, which is why we don't need to specify 1> to redirect standard output: (date 1> now.txt = date > now.txt)

all together

We can redirect multiple streams at once! In this example, we are concatenating two files, redirecting standard output to a file called insects.txt, and redirecting standard error to a file called error.txt.

cat bees.txt ants.txt > insects.txt 2> error.txt
getting fancy

If we wanted to redirect both standard output and standard error to the same file, we could do ls docs > output.txt 2> output.txt

Or we could instead use 2>&1 which is a fancy syntax for saying "redirect standard error to the same location as standard output.

ls docs > output.txt 2>&1
getting fancier

Newer versions of bash also support a fancier syntax for redirecting both standard output and standard error to the same file: the &> notation

ls docs &> output.txt
Ali
  • 922
  • 1
  • 9
  • 24
6

I generally use it with -f option as well, when I am debugging the issue

docker logs -f nginx 2>&1 | grep "127."

It will show us, what we are expecting in real-time.

To include timestamps, add -t

docker logs -ft nginx 2>&1 | grep "127."
Brian Burns
  • 20,575
  • 8
  • 83
  • 77
Nikunj Trivedi
  • 81
  • 1
  • 4
4

Run following command to extract container name for image nginx -

docker ps --filter ancestor=nginx

Copy container ID from last command & then extract log path for your container through below command

grep "127." `docker inspect --format={{.LogPath}} <ContainerName>`
Abhishek Jain
  • 3,815
  • 2
  • 26
  • 26
1

First, use this command ( b1e3c456f07f is the container id ):

docker inspect --format='{{.LogPath}}' b1e3c456f07f

The result will be something like this:

/var/lib/docker/containers/b1e3c456f07f2cb3ae79381ada33a034041a10f65174f52bc1792110b36fb767/b1e3c456f07f2cb3ae79381ada33a034041a10f65174f52bc1792110b36fb767-json.log

Second, use this command ( you can use vim if you like ):

nano /var/lib/docker/containers/b1e3c456f07f2cb3ae79381ada33a034041a10f65174f52bc1792110b36fb767/b1e3c456f07f2cb3ae79381ada33a034041a10f65174f52bc1792110b36fb767-json.log
Dragi Postolovski
  • 396
  • 3
  • 17
  • shouldn’t it be `grep /var/lib/docker/… '127.'` instead of `nano`? – karlsebal Mar 19 '21 at 12:37
  • OC. You could even use MS Word or send it to your printer;) At least you should give advise how to search for the string, as this is the task to be solved – karlsebal Mar 21 '21 at 07:47
0

The top-rated solution didn't work for me - grep printed this output and I couldn't properly find what I was searching for (I'm trying this with a custom docker container, not with nginx):

docker logs collector 2>&1 | grep "ERROR"
grep: (standard input): binary file matches

The solution for me was to use grep -a 'pattern'.

from grep's man page:

-a, --text Process a binary file as if it were text; this is equivalent to the --binary-files=text option.

So this works properly for me:

docker logs collector | grep -a 'ERROR'
Kurt
  • 353
  • 4
  • 14