What's the name lookup differences between curl and nslookup

Xiaoming picture Xiaoming · Jan 19, 2018 · Viewed 12.7k times · Source

I'm lost when trying to figure out when a DNS query will be timed out. Tried multiple scenarios(on Linux):

  1. No name server configured in /etc/resolv.conf

    ###################### curl #######################
    WRITE_OUT="%{http_code}\t%{time_namelookup}\t%{time_connect}\t\t%{time_starttransfer}\t\t%{time_total}\n"
    
    time curl -k -w "$WRITE_OUT" https://www.google.com/
    000     0.000   0.000           0.000           0.000
    
    curl: (6) Could not resolve host: www.goole.com; Unknown error
    
    real    0m0.009s
    user    0m0.000s
    sys     0m0.006s
    
    ##################### nslookup ####################
    time nslookup www.google.com
    ;; connection timed out; trying next origin
    ;; connection timed out; no servers could be reached
    
    real    0m24.012s
    user    0m0.004s
    sys     0m0.009s
    

    As we can see, curl returns immediately(9ms), while nslookup takes much longer(24s). This makes me very confused, curl's behavior makes more sense as there is no name server specified on the host.

  2. Add an unreachable host IP in /etc/resolv.conf, cannot ping to simulate name server down scenario

    ###################### curl #######################
    time curl -k -w "$WRITE_OUT" https://www.google.com/
    000     0.000   0.000           0.000           19.529  
    curl: (6) Could not resolve host: www.goole.com; Unknown error
    
    real    0m20.535s
    user    0m0.003s
    sys     0m0.005s
    
    ##################### nslookup ####################
    time nslookup www.google.com
    ;; connection timed out; trying next origin
    ;; connection timed out; no servers could be reached
    
    real    0m20.008s
    user    0m0.006s
    sys     0m0.003s
    

    Hurray! Looks like curl and nslookup on the same page.

  3. Add an host IP address which can be pinged, but without DNS services, to simulate server is alive but name server service is down

    ###################### curl #######################
    time curl -k -w "$WRITE_OUT" https://www.google.com/
    000     0.000   0.000           0.000           4.513
    curl: (6) Could not resolve host: www.goole.com; Unknown error
    
    real    0m5.520s
    user    0m0.004s
    sys     0m0.005s
    
    ##################### nslookup ####################
    time nslookup www.google.com
    ;; connection timed out; trying next origin
    ;; connection timed out; no servers could be reached
    
    
    real    0m20.010s
    user    0m0.006s
    sys     0m0.005s
    

    Confused again!

The most confusing part is, from the Manual page of resolv.conf, we can that the default value of timeout is 5 seconds, and attempts is 2 times. So I suppose the timeout should be 5 seconds * 2 = 10 seconds. But...confusing...

Edit: Tried again with modifying /etc/nsswitch.conf, only dns method is used. hosts: dns

Scenario 1:

###################### curl #######################
time curl -k -w "$WRITE_OUT" https://www.google.com/
000     0.000   0.000           0.000           0.000
curl: (6) Could not resolve host: www.google.com; Unknown error

real    0m0.051s
user    0m0.004s
sys     0m0.002s
##################### nslookup ####################
time nslookup www.google.com
;; connection timed out; trying next origin
;; connection timed out; no servers could be reached

real    0m24.287s
user    0m0.005s
sys     0m0.014s
######################## dig ######################
time dig www.google.com

; <<>> DiG 9.9.4-RedHat-9.9.4-51.el7 <<>> www.google.com
;; global options: +cmd
;; connection timed out; no servers could be reached

real    0m18.041s
user    0m0.005s
sys     0m0.005s

Scenario 2:

time curl -k -w "$WRITE_OUT" https://www.google.com/
000     0.000   0.000           0.000           19.527
curl: (6) Could not resolve host: www.google.com; Unknown error

real    0m20.533s
user    0m0.003s
sys     0m0.004s

time nslookup www.google.com
;; connection timed out; trying next origin
;; connection timed out; no servers could be reached

real    0m20.009s
user    0m0.005s
sys     0m0.005s

time dig www.google.com
; <<>> DiG 9.9.4-RedHat-9.9.4-51.el7 <<>> www.google.com
;; global options: +cmd
;; connection timed out; no servers could be reached

real    0m15.008s
user    0m0.005s
sys     0m0.003s

Scenario 3:

time curl -k -w "$WRITE_OUT" https://www.google.com/
000     0.000   0.000           0.000           4.512
curl: (6) Could not resolve host: www.google.com; Unknown error

real    0m5.518s
user    0m0.004s
sys     0m0.003s

time nslookup www.google.com
;; connection timed out; trying next origin
;; connection timed out; no servers could be reached

real    0m20.009s
user    0m0.005s
sys     0m0.005s

time dig www.google.com

; <<>> DiG 9.9.4-RedHat-9.9.4-51.el7 <<>> www.google.com
;; global options: +cmd
;; connection timed out; no servers could be reached

real    0m15.009s
user    0m0.005s
sys     0m0.005s

dig has its own timeout mechanism, timeout(5s) * retries(3) = 15s.

Answer

Neil Anuskiewicz picture Neil Anuskiewicz · Jan 19, 2018

nslookup and similar tools query the DNS directly, whereas curl checks the local /etc/hosts first then queries the DNS. So that could be a clue to the issue at hand.

I've read that nslookup's deprecated. Do you have access to dig?