Why does curl and wget take so long to resolve locally mapped DNS on OSX?

Question

I work with a team to develop a web service and client, each of us, of course, working on our own machines. To keep things simple and more consistent, we use similar domains in the code and update /etc/hosts to resolve those domains to localhost.

1. works fine for my web browser; response times are snappy
2. curl and wget take ~4 seconds to resolve DNS before successfully completing the request

I did find the --resolve flag for curl which resolves the delay, but I could just as well use 127.0.0.1 and define the headers necessary to get the same effect.

with flag (and updated /etc/hosts file)

# /etc/hosts
...
127.0.0.1 mp-api.example.local

command

curl -k -o /dev/null -s -w %{time_connect}:%{time_starttransfer}:%{time_total}\\n \
  --resolve mp-api.example.local:8094:127.0.0.1 \
  -H 'Host: mp-api.example.local:8094' \
  'https://mp-api.example.local:8094/api/categories/tree.json'

with 127.0.0.1

curl -k -o /dev/null -s -w %{time_connect}:%{time_starttransfer}:%{time_total}\\n \
  -H 'Host: mp-api.example.local:8094' \
  'https://127.0.0.1:8094/api/categories/tree.json' 

What are some of the reasons for the delay in DNS resolution for tools like curl and wget? And what are some effective ways of troubleshooting that delay?

Answer 1

Per suggestion that there might be settings in the Network tool, I found the values in "Bypass proxy settings..." interesting.

It would appear that I have been having problems with DNS resolution for curl and wget because I have been using a domain ending with .local. Perhaps it's essentially a reserved value.

So I changed my settings so I now use a domain ending with .loc and all is well again -- no more 4 seconds off in la-la land waiting for DNS resolution.

Answer 2

Check /etc/nsswitch.conf file

There configuration how names to id converted. One of lines for hostnames:

hosts:      files mdns4_minimal [NOTFOUND=return] dns myhostname mymachines

Ensure, "files" on first place.

UPD.

Curl local host names on Mac OS X Yosemite

look like curl take responsibility for name resolving and prefer ipv6

try add --ipv4 key or add ipv6 to hosts