I was running into issues with newly upgraded OS Leopard servers where SSH and SFTP were taking a really long time to display the password prompt when you tried to login about 25-30 seconds. I found the following link that describes the issue: http://egopoly.com/2008/02/04/ssh-slow-on-leopard
This lead me to write this post about SRV records. Some of the following information is taken from the URI’s below:
http://www.pantz.org/software/bind/srvdnsrecords.html and http://www.igniterealtime.org/community/thread/39231
When I started my first real sysadmin job back in 1999 I had the simple job of setting up a backup web server. It was going to live in our data center and the primary was going to live at the main office. After setting up the backup web server I thought about how I was going to have it fail over in case the primary died. Thinking about it for a while I thought about how it’s done with NS and MX records. MX for e-mail and NS for the DNS servers themselves. Both work by providing a list of servers that can be contacted for that particular service. Not knowing a ton about DNS my reasoning went that there must be one for http. Reading the O’Reilly DNS and BIND book I found out that this was not the case. There was no equivalent www (for example) record in DNS for HTTP like there was for e-mail. You just had to setup an A record and that was that.
Upon reading further into the DNS and BIND book I found out about DNS SRV records. After reading about the SRV record type it sounded exactly like what I needed. It was a cheap and simple way to have multiple servers running a service and failing over when one server was down. Like I said above there are only 2 records that can provide fail over if one of those services goes down. For any other service your running you have to have some other software doing the checking of the server to see if it’s down or up. If one if the servers in the list goes down then the software you setup can take the machine out of the pool of servers you have. This can be done with a front end proxy (like pound, Apache, or hoststated on OpenBSD) for web servers. The thing is setting up front end watchers and traffic balancers is a bit complicated and a pain. If you just want simple a setup and don’t really have the money to setup anything else SRV records would be the way to go. Let me explain a little more about what SRV records are and why they are so useful.
The SRV resource record was introduced in RFC 2052 (superseeded by RFC 2782). The RFC describes it best so I’ll just quote it “The SRV RR allows administrators to use several servers for a single domain, to move services from host to host with little fuss, and to designate some hosts as primary servers for a service and others as backups”. What does this mean? It means we can take any service like http or ftp and spread them across as many servers as you want. Then you can designate which server takes priority over others, distribute load, and provide backup services if a service fails.
An example of what an SRV record structure looks like is the in the following example:
# Service.Proto.Name Class SRV Priority Weight Port Target
_http._tcp.www.pantz.org. IN SRV 0 2 80 www.pantz.org.
IN SRV 0 1 80 www2.pantz.org.
IN SRV 1 1 8080 www3.pantz.org.
In the record above requests for the website http://www.pantz.org will go to port 80 on http://www.pantz.org and www2.pantz.org. The host http://www.pantz.org will get twice the queries that www2.pantz.org gets. If both hosts go down, the queries will go to www3.pantz.org on port 8080. For a better detailed explanation of SRV records see RFC 2782.
How fantastically simple is that! All in a DNS record. This is a record almost all DNS servers support right now. You can put in a record for any service you want to run (FTP,IMAP,POP,etc) and you get all that great stuff. So what’s the problem? The problem is (as I found out back in 1999 when first learning about SRV records) almost no one uses them. Can you f’ing believe that!
For SRV records to be useful the client software (web browser,ftp client, etc) has to be able to read the record from DNS and use the information the DNS server gives it. All client software queries DNS for A records. Why the hell can’t it ask for an SRV record also? All it would take is a little code on the client side to parse the SRV info and little logic to figure out which server to contact.
Almost 10 years from when I first read about SRV records to now and most people still don’t use SRV records. No web browser supports it. Mozilla has had a bug filed to get one since 1999. How sad is that? Someone has written a patch but they don’t know what to do with it. No email client uses them either. How great would that be for backup POP or IMAP servers? With SRV records you could almost get rid of the MX record. The only things that use SRV records now are SIP,LDAP,XMPP,KERBEROS, and a few others. That is not much.
So why don’t people want to use them? I my opinion we have a catch-22 going on here. DNS admins won’t put in the records because the clients won’t use them. The clients won’t write the code to use the SRV records because the DNS admins won’t put in records. Someone has to step up here.
Some will try to argue that this is one more query to the DNS server and it will take more time to do the lookup. I’m here to tell you the the small amount of time it takes to do this look up is outweighed by the advantages these records provide. I would rather endure waiting a little longer (we are talking low 100’s of milliseconds here) for a web page to load because it had to look up the SRV records than for the website not to load because the person running the site did not have any easy way of setting up a fail over server. Just like what happened to me in my first sys admin job back in 1999. Because I found out that no one was using SRV records I had to setup the backup server and just changed the A record by hand when the machine went down. So inefficient.
SRV records also provide a way to run a service on an alternate port. This would be a godsend for people who try to run services on their home machines. Many ISP’s block popular incoming ports but leave open many others. So an ISP that blocks port 80 in (I’m looking at you Verizon FIOS) could easily be worked around with SRV records. Just specify an alternate port in your SRV record like in the example above and your done. Same goes for any service that they block in. If we started using SRV records instead of MX records for mail then we could run SMTP on any port we wanted. It’s all spelled out in the record.
I have to practice what I preach so I put in SRV records for the web and mail servers for pantz.org. If you want to see what an SRV record looks like you can check for the entry using the host program found on most UNIX systems. I’ll use my domains as an example.
host -t SRV _smtp._tcp.pantz.org
host -t SRV _http._tcp.www.pantz.org
The first command above will show the 2 mail servers that service pantz.org. The primary and the backup. The priority levels are set to reflect this. The second command shows just the one web server for now. These entries took only a few minutes to setup. They really are an easy way to get some very basic fail over and balancing for your servers. Now, if we can just get some clients than would use these SRV records the internet would be a much better place.