I "inherited" old code for a server running several docker containers. Grafana and Prometheus are used to monitor stats, as well as node-exporter. Unfortunately, the node-exporter container is shown as down in Prometheus. The error message is Get "http://IP:9101/metrics": context deadline exceeded
The node-exporter container is the only one with network_mode host, the rest of the containers is in a user-defined bridge network. When I try to curl the metrics endpoint from the host machine with curl localhost:9091/metrics, it works. In the prometheus.yml, the scrape_config for node is defined as follows:
- job_name: node
scrape_interval: 30s
static_configs:
- targets: ['HOST_NETWORK_IP:9101']
I have to get the connection between node_exporter and prometheus running and I have the feeling that the solution is most likely something simple, but I can´t figure it out, as I don´t really have much experience with docker networking. Any help would be appreciated!
Clarifications: The HOST_NETWORK_IP is the IP of the Open Stack Instance. However, normally, queries are routed through traefik.
I am not quite sure where the exporter is listening, but it should be the default. The only change to listening in the docker-compose for the exporter is -web.listen-address=:9101.
I didn´t set it up to run with host networking, so I am not a 100% sure about the reasoning behind it, however from my research it seems to me that this is necessary to get metrics from the host system.