0

Trying to write a script to parse a list of domains to confirm their registrars are being updated correctly. I run the whois against a domain name and all I want are the lines that START with "Name Server" or "Domain Name"

When I run and grep the 'whois' output from the shell, it works perfectly:

$ whois customerdomain.com | egrep -e ^'Name Server|^Domain Name'
Domain Name: CUSTOMERDOMAIN.COM
Name Server: ns.buydomains.com
Name Server: this-domain-for-sale.com

But when I take a list of domains and put them in a shell script, the "or" operator is ignored and the "Domain name" fails to be included in the output:

Example script (whois-list.sh):

#!/bin/bash
whois customerdomain.com | egrep -e ^'Name Server|^Domain Name'
whois contoso.com | egrep -e ^'Name Server|^Domain Name'
whois google.com | egrep -e ^'Name Server|^Domain Name'
whois microsoft.com | egrep -e ^'Name Server|^Domain Name'

Output:

$ ./whois-list.sh
Name Server: ns.buydomains.com
Name Server: this-domain-for-sale.com
Name Server: ns1-205.azure-dns.com
Name Server: ns3-205.azure-dns.org
Name Server: ns2-205.azure-dns.net
Name Server: ns4-205.azure-dns.info

I've tried like 10 different variations that all seem to work fine at the prompt, but always seem to dump the 'or' operator when run from the script.

Ashmatash
  • 3
  • 1

1 Answers1

2

Hypothesis

Your whois-list.sh contains Windows/DOS line endings (denoted CRLF or \r\n) instead of Unix line endings (denoted LF or \n).

The extra character (denoted CR or \r) is not recognized by the shell interpreting the script as a part of the line terminator; it is recognized as a part of input, so it gets "glued" to the last argument passed to egrep.

In effect egrep does not see …|^Domain Name, it sees …|^Domain Name\r where \r denotes the character you did not mean to pass to egrep. The output from whois never matches this pattern.

To confirm

If file whois-list.sh tells you … with CRLF line terminators then this is the case.

Alternatively and without additional tools (like file), you can try to put a line like this into the script:

whois customerdomain.com | egrep -e ^'Name Server|^Domain Name' #

where everything after # (including \r, if any) will be seen as a comment. If this line works and the same line without # misbehaves then it's a strong clue \r is there.

In general not every possible line of shell code can be "fixed" this way. This is not a good fix, just a simple way to diagnose the problem.

What about shebang?

In Linux a trailing \r after #!/bin/bash would make the shebang misbehave in the first place. Either this line alone is fine in your script, or Cygwin does some fixing here.

Fix

A common tool to convert files from Windows/DOS line endings to Unix line endings is dos2unix:

dos2unix whois-list.sh

(Other possibilities: here). Some GUI text editors can convert the file for you, even in Windows (e.g. Notepad++). Fixing the line endings in whois-list.sh is the right thing to do.

Do not use Windows-centric text editors to edit scripts for Cygwin (or Linux/Unix). Make sure the editor you use can and does save your scripts with Unix line endings.

Kamil Maciorowski
  • 81,893