1

I'm composing yaml file for scripts running in docker and orchestrated by kubernetes. Is there a way to evaluate the resource utilization for a specicific command or docker, or what's the best practice to set the limit of cpu and mem for pods?

Edit

Most of these scripts will run in a short time, so it's hard to get the resource info. I just wanna try to find a tool to get the maximum usage of cpu and mem, the tool works like time, to print out the execution time.

HoldOffHunger
  • 18,769
  • 10
  • 104
  • 133
Yang
  • 759
  • 2
  • 9
  • 30

2 Answers2

3

You can view statistics for container(s) using the docker stats command.

For example;

docker stats containera containerb

CONTAINER           CPU %               MEM USAGE / LIMIT     MEM %               NET I/O               BLOCK I/O
containera          0.00%               24.15 MB / 1.041 GB   2.32%               1.8 MB / 79.37 kB     0 B / 81.92 kB
containerb          0.00%               24.95 MB / 1.041 GB   2.40%               1.798 MB / 80.72 kB   0 B / 81.92 kB

Or, see processes running in a container using docker top <container>

docker top containera
UID                 PID                 PPID                C                   STIME               TTY                 TIME                CMD
root                4558                2850                0                   21:13               ?                   00:00:00            sh -c npm install http-server -g && mkdir -p /public && echo "welcome to containera" > /public/index.html && http-server -a 0.0.0.0 -p 4200
root                4647                4558                0                   21:13               ?                   00:00:00            node /usr/local/bin/http-server -a 0.0.0.0 -p 4200

Limiting resources

Docker compose (like docker itself) allows you to set limits on resources for a container, for example, limiting the maximum amount of memory used, cpu-shares, etc.

Read this section in the docker-compose yaml reference, and the docker run reference on "Runtime constraints on resources"

thaJeztah
  • 27,738
  • 9
  • 73
  • 92
  • Thanks! Most of the dockers are running in a short time, and I think it's hard to monitor through tools like top or ps. – Yang Jan 18 '16 at 02:39
  • I've read the "Runtime constraints on resources", it's useful, but I'm curious about how to set a *appropriate* limitation. Thanks in advance. – Yang Jan 18 '16 at 03:00
  • `docker stats` obtains it's data via the docker remove API [using this endpoint](https://docs.docker.com/engine/reference/api/docker_remote_api_v1.21/#get-container-stats-based-on-resource-usage). If you're automating things, you may be able to collect those stats and store them in, for example, an [ELK](http://nathanleclaire.com/blog/2015/04/27/automating-docker-logging-elasticsearch-logstash-kibana-and-logspout/) stack. Or use [cAdvisor / Elasticsearch / Grafana / Influxdb](http://blog.tutum.co/2014/08/25/panamax-docker-application-template-with-cadvisor-elasticsearch-grafana-and-influxdb/) – thaJeztah Jan 18 '16 at 03:01
  • > "I'm curious about how to set a appropriate limitation." That's a tough one to answer, because that really depends on your use case. It's really advisable to *always* set a memory-limit on a container, to prevent a container using up all available memory of the host, and (potentially) resulting in the Kernel "OOM" killing off random processes. – thaJeztah Jan 18 '16 at 03:04
2

There are some good answers in the question: Peak memory usage of a linux/unix process

TL;DR: /usr/bin/time -v <command> or use valgrind.

That should help you get an idea for how much memory you need to assign as the limit for your app but CPU is a bit different beast. If your app is CPU bound then it will use all the CPU you give it no matter what limit you set. Also, in Kubernetes you assign cores (or millicores) to apps so it's not always terribly useful to know what % of the CPU was used on any particular machine as that won't readily translate to cores.

You should give your app as many CPU cores as you feel comfortable with and that allows your app to succeed in an acceptable amount of time. That will depend on cost and how many cores you have available in your cluster. It also depends a bit on the architecture of your app. For instance, if the application can't take advantage of multiple cores then there isn't much use in giving it more than 1.

In case you have any longer running apps, you could try installing Kubedash. If you have Heapster installed then Kubedash uses the built in metrics for Kubernetes to show you average and max CPU/Memory utilization. It helps a lot when trying to figure out what requests & limits to assign to a particular application.

Kubedash screenshot

Hope that helps!

Community
  • 1
  • 1
Ian Lewis
  • 917
  • 1
  • 7
  • 17