7

This is a strange problem I've spent all day on. It would be great if someone could shed some light on this.

The problem exhibits as a problem with name resolution, but I'm not sure that's the root cause:

# host www.google.com
;; connection timed out; no servers could be reached

so far so boring, but wait!:

#systemd-resolve www.google.com
www.google.com: 209.85.202.103
                209.85.202.106
                209.85.202.105
                209.85.202.104
                209.85.202.147
                209.85.202.99

Simple, the problem is with resolv.conf right?

# This file is managed by man:systemd-resolved(8). Do not edit.
#
# 127.0.0.53 is the systemd-resolved stub resolver.
# run "systemd-resolve --status" to see details about the actual nameservers.
nameserver 127.0.0.53

search xxx.uk xyz

So the system is using the systemd resolver?

#dig @127.0.0.53 www.google.com
; <<>> DiG 9.10.3-P4-Ubuntu <<>> @127.0.0.53 www.google.com
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached

ok, so if the systemd resolver says it's on 127.0.0.53 why is it not responding. 

#sudo netstat -lupn | grep 127
udp        0      0 127.0.0.53:53           0.0.0.0:*                           1679/systemd-resolv

If it's not listening, what is systemd-resolv doing?

Global
          DNSSEC NTA: 10.in-addr.arpa
                      16.172.in-addr.arpa
                      168.192.in-addr.arpa
                      17.172.in-addr.arpa
                      18.172.in-addr.arpa
                      19.172.in-addr.arpa
                      20.172.in-addr.arpa
                      21.172.in-addr.arpa
                      22.172.in-addr.arpa
                      23.172.in-addr.arpa
                      24.172.in-addr.arpa
                      25.172.in-addr.arpa
                      26.172.in-addr.arpa
                      27.172.in-addr.arpa
                      28.172.in-addr.arpa
                      29.172.in-addr.arpa
                      30.172.in-addr.arpa
                      31.172.in-addr.arpa
                      corp
                      d.f.ip6.arpa
                      home
                      internal
                      intranet
                      lan
                      local
                      private
                      test

Link 20 (veth10858e2)
      Current Scopes: LLMNR/IPv6
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no

Link 14 (vnet0)
      Current Scopes: LLMNR/IPv6
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no

Link 13 (virbr0-nic)
      Current Scopes: none
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no

Link 12 (virbr0)
      Current Scopes: LLMNR/IPv4
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no

Link 11 (docker0)
      Current Scopes: none
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no

Link 10 (docker_gwbridge)
      Current Scopes: LLMNR/IPv4 LLMNR/IPv6
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no

Link 3 (em2)
      Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 192.168.100.1
                      192.168.100.2
                      192.168.100.3
                      192.168.100.4
          DNS Domain: cqp

Link 2 (em1)
      Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: xxx.xxx.x.xx
                      xxx.xxx.x.xx
          DNS Domain: xxx.uk

I have several severs, they're all running up to date ubuntu artful, this problem has moved between servers in the process of trying to fix this.
They are part of a docker swarm and removing, then re-adding fix the problem at one point.

I turned on systemd-resolved debugging but there was no output when a request was made.

Richard Collins
  • 71

3 Answers3

12

I found this answer on Hacker News which suggested symlinking /etc/resolv.conf to /run/systemd/resolve/resolv.conf:

sudo rm /etc/resolv.conf
sudo ln -s /run/systemd/resolve/resolv.conf /etc/resolv.conf

That worked for me. I don't think I had to restart any services after recreating the symlink.

Neil Smith
  • 221
1

It sounds crazy, but you need to allow access to 127.0.0.53 to use systemd-resolved

iptables -I INPUT -s localhost -d 127.0.0.53  -j ACCEPT
Machavity
  • 663
David Ordyan
  • 11
0

Check whether resolv.conf file is available in the file path: /run/systemd/resolve/resolv.conf

It is Symbolic linked with /etc/resolv.conf

If the file is not available then create it in this file location /run/systemd/resolve/resolv.conf and delete the file in /etc/resolv.conf

Now relink the file by cmd sudo ln -s /run/systemd/resolve/resolv.conf /etc/resolv.conf

Now try to ping, it will fix the issue most

Dishone Prabu J
  • 1